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THOMAS  CRAIG  HARRINGTON.  A  Multistage  R&D  Project  Selection  Model 
with  Multiple  Objectives  (Under  the  direction  of  WILLIAM  A.  FISCHER.) 

— This  dissertation  presents  a  multiple  objective,  zero-one 

mathematical  programming  model  for  the  stochastic,  multistage  R&D 

project  selection  and  resource  allocation  problem.  The  model  is 

intended  for  those  research  and  development  activities  which  have 

a  certain  degree  of  autonomy  in  the  selection  of  their  research 

portfolio  and  which  concentrate  efforts  in  applied  research  or 

development  projects.  ~- 

^Each  feature  of  the  proposed  model  is  designed  to  incorporate 
realistic  aspects  of  the  R&D  decision  making  process.  First,  the 
model  includes  an  integer  goal  programming  formulation  which  enables 
the  formal  incorporation  of  multiple,  non commensurate  objectives 
along  with  integer  treatment  of  possible  projects.  Second,  decision 
tree  planning  techniques  are  used  to  represent  the  sequential  aspects 
of  allocating  scarce  resources  over  the  planning  horizon,  to  reflect 
various  project  and  goal  relationships,  and  to  incorporate  proba¬ 
bilistic  future  events  and  outcomes.  Third,  simulation  techniques 
are  used  to  convert  the  probabilistic  parameters  into  deterministic 
inputs  for  the  mathematical  formulation.  Finally,  the  model  includes 
a  heuristic  based  solution  algorithm  involving  system  constraint 
partitioning  procedures,  and  iterative  one-pass  variable  selection 
and  solution  variable  exchange  routines  to  enable  the  efficient 

solution  of  realistic  size  problems,  - 

Example  problems  are  presented  to  examine  the  conceptual  utility 
of  the  proposed  model.  Practical  utility  is  demonstrated  through 
testing  the  model  against  recent  state-of-the-art  project  selection 


techniques.  Finally,  model  feasibility  is  demonstrated  through 
an  actual  application  and  appraised  through  interviews  with  R&D 
administrators . 

Applications  of  the  model  to  other  types  of  resource  allo¬ 
cation  and  capacity  management  problems  are  proposed  as  areas  for 


further  research. 
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CHAPTER  I 


INTRODUCTION 

"It  should  be  obvious  that  the  perform¬ 
ance  of  a  research  organization  can  be 
no  better  than  the  projects  selected  as 
inputs  to  the  R&D  system.  Working  on  the 
wrong  project  is  wasteful,  and  the  result¬ 
ing  penalty  is  a  low  return  on  R&D  investment. 
No  wonder,  then,  that  project  selection  is 
perhaps  the  most  critical  step  of  the  entire 
R&D  process . " 


(Introduction  to  a  Collection  of  Papers 
on  Project  Selection,  Research  Manage¬ 
ment,  Vol.  14,  September,  1971,  p.  24) 

The  R&D  project  selection  decision  is  concerned  with  the  alloca¬ 
tion  of  scarce  organizational  resources  such  as  money,  skills,  and 
equipment,  among  a  set  of  proposals  for  performing  research  or  devel¬ 
opment  projects.^*  The  selection  decision  is  important  to  sponsoring 
and  performing  agencies  since  the  portfolio  of  projects  selected  typ¬ 
ically  represents  considerable  risk  and  investment.  To  illustrate 
the  magnitude  of  expenditures  in  aggregate  terms,  U.S.  Government 
agencies,  industries,  universities  and  nonprofit  enterprises  invested 
$  40.8  billion  in  R&D  projects  during  1977  and  are  expected  to  invest 
$  45.2  billion  in  1978. 3 

The  selection  decision  is  a  complex  and  difficult  process  due  to 
the  nature  of  the  R&D  environment.  Generally,  the  number  of  project 
proposals  together  with  existing  projects  which  must  be  considered, 
make  demands  on  resources  in  excess  of  current  and  forecasted 
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capabilities;  simply  stated,  most  R&D  activities  are  faced  with 
more  proposals  for  projects  than  resources  can  support  at  any  given 
time.  Furthermore,  R&D  projects  must  be  evaluated,  selected  and 
measured  against  multiple  and  often  conflicting  economic  and  non¬ 
economic  objectives.  In  addition,  the  very  nature  of  R&D  to  push 
forward  the  existing  scientific  and  technological  state-of-the-art 
introduces  uncertainty  into  the  project  selection  process. ^  Thus, 
proposal  estimates  for  key  items  such  as  the  probabilities  of 
technical  and  commercial  success,  timing  and  levels  of  income  and 
cost  streams,  project  duration,  resource  usage  rates  and  resource 
availabilities  can  be  difficult  to  accurately  determine. 

Many  qualitative  and  quantitative  techniques  have  been  developed 
in  the  past  twenty-five  years  to  assist  R&D  managers  in  the  project 
selection  and  resource  allocation  decision.  In  a  1964  survey,  Baker 
and  Pound^  cited  eighty-two  sources  in  the  literature  directly  related 
to  project  selection;  ten  specific  models  were  reviewed  as  being  repre¬ 
sentative  of  the  more  than  fifty  techniques  available.  Cetron,  Martino 

and  Roepcke^  identified  and  evaluated  thirty  techniques  in  their  1967 

8 

survey.  In  1971,  Gear,  Lockett  and  Pearson  reviewed  nine  mathematical 
programming  models  dealing  with  the  portfolio  selection  and  resource 

Q 

allocation  problem.  Souder  singled  out  forty-one  models  as  being 
representative  of  the  more  than  one  hundred  techniques  known  to  exist 
by  1972.  In  that  same  year.  Baker  and  Freeland‘S  provided  a  compre¬ 
hensive  assessment  of  the  state-of-the-art  models.  Their  findings 
echoed  those  of  the  other  reviewers  and  practitioners:  while  some 
excellent  techniques  have  been  developed  to  address  specific  factors 
of  the  problem,  existing  models  are  not  being  used  by  research  and 
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development  managers  because  of  key  limitations  such  as: 

1)  Inadequate  treatment  of  multiple  objectives,  including 
both  economic  and  noneconomic  criteria. 

2)  Inadequate  treatment  of  the  time  variant  property  of  data 
and  criteria. 

3)  Failure  to  consider  the  experience  and  knowledge  of  R&D 
managers,  scientists  and  engineers. 

4)  Inability  to  represent,  establish  and  maintain  balance  in 
the  R&D  program. 

5)  Inadequate  treatment  of  risk  and  uncertainty. 

6)  Failure  to  represent  project  interrelationships  with  respect 
to  both  value  contribution  and  resource  utilization. 

Interviews  conducted  with  a  small  but  representative  sample  of  organi¬ 
zations  involved  in  R&D  activities  as  part  of  this  research  have 
found  agreement  with  these  findings  of  other  researchers  that  quantita¬ 
tive  project  selection  models  are  not  being  used  in  the  field.  The 
most  significant  reason  for  this  lack  of  application  is  indeed  the 
failure  of  the  model  builders  to  account  for  realism  with  respect  to 
multiple  objectives,  multiple  stages  in  project  evolution  and  uncer¬ 
tainty.  However,  three  additional  reasons  were  identified  during  the 
interview  sessions  which  provide  additional  insight  into  the  R&D  deci¬ 
sion  making  process.  First,  in  the  case  of  the  R&D  laboratories  of  a 
large  textile  firm  and  a  pharmaceutical  company,  there  is  a  lack  of 
awareness  of  management  science  models  proposed  as  decision  aiding 
tools.  Second,  in  the  case  of  a  nonprofit  research  institute  and  the 
research  laboratory  of  a  government  regulatory  agency,  project  selec¬ 
tion  occurs,  to  some  extent,  at  the  client  or  sponsoring  activity  level 
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rather  than  within  the  research  organization  and  the  R&D  organization 
has  little  alternative  but  to  accept  mandated  projects."^  Third, 
especially  in  the  case  of  the  pharmaceutical  company,  project  proposal 
data,  in  the  laboratory  visited,  is  not  collected  with  the  detail 
necessary  for  use  of  quantitative  selection  models.  It  should  be 
noted  that  this  research  laboratory  is  principally  engaged  in  explora¬ 
tory  research  for  the  advancement  of  medicine  knowledge  rather  than  in 
applied  research  or  development  work  associated  with  specific  end  use 
items,  and  because  project  proposals  for  exploratory  research  are  by 
their  very  nature,  ill  defined,  it  appears  that  formal  selection  tech¬ 
niques  have  limited  applicability  in  this  type  of  environment. 

In  view  of  the  limitations  or  the  utility  of  the  present  state- 
of-the-art  models,  the  objective  of  this  research  is  stated  as 
follows : 

Develop  a  workable,  multicriteria  mathematical  pro¬ 
gramming,  format  for  selecting  interdependent  R&D 
projects  over  a  multiperiod  planning  horizon. 

The  managers  and  directors  of  the  R&D  organizations  interviewed 
during  the  course  of  this  research  gave  particular  encouragement 
to  the  pursuit  of  this  objective  and  contributed  towards  the  follow¬ 
ing  statement  of  the  scope  of  model  applications: 

The  project  selection  and  resource  allocation  model 
is  designed  and  intended  for  those  R&D  activities 
which  have  a  certain  degree  of  autonomy  in  the  selec¬ 
tion  of  their  research  portfolio,  and  which  concen¬ 
trate  efforts  on  applied  research  or  development 
projects  associated  with  clearly  defined  end  uses. 
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The  following  section  serves  to  amplify  on  the  purpose,  scope  and 
contribution  of  this  research. 

Purpose  and  Scope  of  This  Research 

In  order  to  establish  the  contribution  of  this  research,  it 

is  important  to  view  the  project  selection  and  resource  allocation 

decision  as  an  integrated  part  of  a  larger  process  that  involves 

idea  generation  and  handling,  project  proposal  evaluation,  project 

initiation  and  control,  and  project  completion  and  termination. 

12  13  1 

Descriptive  studies  by  Dean,  Souder,  Beattie  and  Reader,  and 
Brandenburg^  discuss  these  essential  features  and  are  used,  in 
part,  to  diagram  the  total  project  selection  and  evaluation  process 
as  shown  in  Figure  1-1.  In  addition,  steps  concerned  with  estab¬ 
lishing  research  program  objectives  (boxes  1  and  2),  and  their  rela¬ 
tive  priorities  (boxes  3  and  4),  as  well  as  a  representation  of 
component  interactions  through  feedback  mechanisms,  have  been  added 
to  embed  the  research  in  a  realistic  setting.  As  a  first  step,  the 
organization  develops  objectives  for  the  research  program  and  priori¬ 
tizes  these  objectives  (boxes  1  through  4).  Comparative  techniques 
such  as  0-Sort  and  pairwise  comparison  by  forced  choice  have  been 

1  A 

applied  in  these  decision  stages.  The  next  series  of  steps  involve 

idea  generation  and  proposal  development  (boxes  5  through  8). 

Qualitative  techniques  such  as  scoring  methods  and  economic  models 

have  been  used  in  these  stages  to  construct  indexes  of  project  worth. 

Although  the  above  techniques  have  been  suggested  for  the  project 

1 8 

selection  decision  stage  by,  for  instance,  Souder,  Moore  and 

19  ancj  Villers,“^  because  of  their  inability  to  simultaneously 
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Figure  1-1 
The  R&D  Process 
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consider  resource  constraints  (box  9),  these  models  are  likely  to  gen¬ 
erate  portfolios  with  a  sub-optimal  allocation  of  resources.  Quantita¬ 
tive  techniques  based  on  mathematical  programming  have  been  developed 

for  the  project  selection  and  resource  allocation  stage  (box  10)  by, 

21  22 

for  example,  Asher  and  Beged-Dov,  but  are  restricted  in  their 
utility  because  they  tend  to  allocate  resources  to  a  portfolio  of 
projects  by  optimizing  a  single  or  unidimensional  objective  function. 
Further,  none  of  the  quantitative  techniques  surveyed,  including  those 
that  address  non commensurate  multiple  criteria,  are  designed  to  be 
integrated  into  the  total  R&D  process  as  either  the  multistage  nature 
of  discrete  projects  and  multiple  resource  constraints  are  ignored 
and/or  uncertainty  in  the  project  proposal  data  parameters  is  not 
considered.  The  purpose  of  this  research  is  therefore  to  fill  a  void 
in  the  selection  process  through  development  of  a  multistage,  multiple 
objective  mathematical  model  for  the  box  10  decision  stage. 

It  is  further  proposed  that  the  model  is  useful  as  an  analytic 
technique  to  interact  with  the  resource  determination,  proposal 
development,  and  multiple  objective  generation  stages.  As  an  example, 
even  if  economic  and  noneconomic  criteria  can  be  established  within 
an  F&D  group,  it  remains  unlikely  that  consensus  can  be  reached  on 
the  relative  importance  of  the  group’s  goals,  especially  in  a  hier¬ 
archical  organization  structure.  If  consensus  remains  blocked  after 
several  iterations  through  the  decision  process  represented  by  boxes 
3  and  4  of  Figure  1-1,  the  group  has  been  divided  into  proponents  of 
two  or  more  conflicting  priority  structures.  At  this  point,  it  is 
suggested  that  the  present  model  be  used  to  generate  portfolios  of 
projects  for  each  set  of  priority  rankings.  Since  the  model  provides 
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the  decision  makers  with  additional  information  regarding  goal  attain¬ 
ment  levels  for  corresponding  portfolios  and  priority  structures, 

/ 

consensus  could  be  facilitated  through  trade-off  analysis  or  identi- 

/ 

fication  and  evaluation  of  dominant  portfolios  and/or  projects.  Simi¬ 
larly,  decision  makers  responsible  for  establishing  resource  levels 

mav  use  the  model  to  analyze  multiple  goal  attainment  levels  associ- 

/ 

ated  with  portfolios  generated  for  various  resource  constraint  sets. 
Alternatively,  changes  in  resource  levels  over  the  planning  horizon 
may  be  suggested  in  selection  ?f  p  particular  portfolio  provided  that 
some  resource  flexibili*  exists.  Finally,  decision  makers  responsible 
for  estimating  probaoilistic  outcomes  (boxes  7  and  8,  Figure  1-1)  could 
benefit  from  using  the  ^odel  to  generate  alternative  portfolios  for 
dominant  futures  or  for  various  changes  in  the  estimates.  Statistical 
analysis  and  group  interactions  could  then  be  used  in  searching  for 
dominant  portfolios  and/or  projects  for  construction  of  the  final 
research  effort. 

In  summary,  the  contribution  of  this  research  effort  is  a  multi¬ 
stage,  multiple  objective  R&D  project  selection  model  designed  to 
interact  with  portions  of  the  larger  R&D  project  selection  and  evalu¬ 
ation  system,  KTiile  the  research  focuses  on  the  project  selection 
problem,  the  model  is  also  adaptable  for  the  termination  decision 
(boxes  15  and  16,  Figure  1-1).  That  is,  in  addition  to  new  proposals 
for  the  R&D  effort,  existing  projects  may  also  be  considered  in  the 
selection  decision  without  constraint  mechanisms  forcing  their 
acceptance  in  the  final  portfolio.  Model  solutions  failing  to  accept 
an  existing  project  based  upon  the  multiple  goal  attainment  levels 
would  suggest  project  termination  prior  to  planned  completion. 


The  steps  involved  with  initiation  of  effort  on  selected  projects, 
project  control,  completion  and  evaluation  of  results  (reference  boxes 
13,  14,  17  and  18,  Figure  1-1)  are  important  stages  in  the  total  pro¬ 
cess  but  are  beyond  the  scope  of  this  research. 


Summary  of  the  Chapters 

Chapter  II  presents  a  survey  of  the  principal  techniques  pro¬ 
posed  in  the  literature  for  the  R&D  project  selection  and  resource 
allocation  problem.  From  this  review,  a  set  of  prescriptions  for 
a  general  analytic  technique  are  developed. 

Chapter  III  presents  the  basic  development  of  the  multistage, 
multiple  objective  model  and  solution  algorithm.  The  complete 
model  includes: 

1)  Integer  goal  programming  formulation  which  enables  the 
formal  incorporation  of  multiple  objectives  along  with 

a  recognition  of  the  need  for  integer  rather  than  linear 
treatment  of  possible  projects. 

2)  Decision  tree  planning  of  R&D  projects  over  a  multiperiod 
planning  horizon.  This  structure  is  used  for  representa¬ 
tion  of  the  sequential  aspects  of  allocating  scarce 
resources  over  the  planning  horizon,  to  reflect  various 
project  and  goal  relationships ,  as  well  as  incorporating 
chance  future  events  and  probabilistic  outcomes. 

3)  Simulation  of  the  chance  future  events  and  project  out¬ 
comes  to  convert  stochastic  parameters  into  deterministic 


inputs  for  the  integer  goal  programming  model. 


4)  Heuristic  based  solution  algorithm  involving  selected 


system  constraint  partitioning  procedures,  and  iterative 
one-pass  variable  selection  and  solution  variable  exchange 
routines  to  enable  the  rapid  solution  of  realistic  size 
problems. 

5)  Statistical  analysis  of  model  results. 

In  Chapter  IV,  a  hypothetical  case  study  is  presented  to 
demonstrate  the  application  potential  of  the  model  for  portfolio 
selection  and  resource  allocation  decision  processes. 

Chapter  V  presents  several  applications  of  the  model,  three 
of  which  represent  validity  and  efficiency  evaluations  through 
comparisons  with  other  state-of-the-art  zero-one,  dynamic  program¬ 
ming,  and  integer  goal  programming  models.  An  actual  application 
for  project  selection  and  resource  allocation  decision  processes 
in  the  R&D  laboratory  of  a  large  textile  firm  is  also  included. 
Finally,  a  summary  of  the  findings  of  the  interview  sessions  with 
the  directors  of  five  R&D  organizations  concerning  additional 
applications  and  evaluation  of  model  feasibility  is  presented. 

Chapter  VI  discusses  conclusions  and  suggests  areas  for 
further  research. 
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CHAPTER  II 


SURVEY  OF  THE  PRINCIPAL  TECHNIQUES 
AND  MODEL  PRESCRIPTIONS 

"It  is  important  to  ensure  that  a  (project  se¬ 
lection)  system  is  set  up  which  does  not  reject 
half  baked  ideas  solely  because  they  lack  baking, 
but  yet  does  not  spend  so  much  effort  on  evalu¬ 
ating  possible  ideas  in  depth  that  there  is 
hardly  any  effort  left  to  do  the  research." 

(Beattie  and  Reader,  Quantitative 
Management  in  R&D,  p.27) 

Prior  to  World  War  II  and  in  the  early  post-war  years,  com¬ 
panies  with  large  enough  R&D  laboratories  could  choose  the  product 
line  or  area  in  which  to  devote  its  P.&D  resources  and  expect  to 
reap  considerable  benefits.^  During  this  period,  Rubenstein 
observed  that  reasonably  efficient  research  endeavors  produced 

benefits  that  far  out-weighed  the  costs  and  thus,  most  companies 

2 

were  not  concerned  with  precise  methods  of  project  selection.  As 
the  post-war  markets  matured  in  the  1950’s,  companies  began  to  see 
a  relationship  between  intensified  R&D  efforts  and  corporate  growth. 
Brandenburg  noted  in  his  1966  descriptive  study ^  that  the  optimism, 
permissiveness  and  faith  of  the  post-World  War  II  industrial  re¬ 
search  revolution  gave  way  to  emphasis  on  measurement  and  control  in 
budgeting  resources  among  alternative  R&D  project  opportunities.  As 
a  result,  formal  project  selection  methods  were  desired  by  R&D  mana¬ 
gers  as  decision  aiding  tools.  In  attempting  to  meet  this  need, 


management  scientists  and  some  practitioners  developed  techniques 
that  can  be  generally  categorized  as  scoring  models,  comparative 
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methods,  economic  index  and  mathematical  programming  models.  These 
are  reviewed  in  the  next  four  sections.  Following  this  survey,  two 
additional  sections  are  devoted  to  reviewing  techniques  designed  to 
incorporate  the  dynamic  nature  and  uncertainty  associated  with  R&D 
projects  and  methods  which  address  the  multiple  criteria  problem. 

Scoring  Models 

Scoring  models  are  among  the  earliest  formal  approaches  to 

addressing  the  multiple  goals  of  an  R&D  group.  Methods  such  as  those 

proposed  by  Dean  and  Nishry,^  Garguilo,  et.  al. Mottley  and  New- 
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ton,  and  Pound,  begin  with  development  of  a  list  of  all  the  cri¬ 
teria  that  are  important  in  choosing  projects.  For  example,  the 
Mottley  and  Newton  model  includes  promise  of  success,  time  to  com¬ 
pletion,  cost  of  the  project,  strategic  need  and  market  gain  as  the 
criteria.  Next,  numerical  scales  are  assigned  to  each  criteria  and 
a  panel  of  experts  are  asked  to  rate  each  project  proposed  on  the 
criterion  scale.  The  resultant  ratings  are  then  combined  in  a  mul¬ 
tiplicative  or  additive  fashion  to  determine  a  project  score  which 
is  used  to  rank  order  the  projects.  The  R&D  portfolio  is  determined 
by  successively  selecting  projects  as  they  appear  in  the  ranking  until 
some  constraint  such  as  budget,  personnel  or  facilities  is  reached. 

Scoring  models  are  relatively  easy  to  develop  and  evidence  sug¬ 
gests  they  have  found  some  use  in  actual  R&D  environments,  particu¬ 
larly  in  the  evaluation  of  project  proposals  and  selection  of  basic 
9  10 

research  projects.  However,  Beattie  and  Reader  noted  that  their 


inherent  simplicity  can  lead  to  an  unwarranted  appearance  of  validity. 


Seiler^  expanded  on  this  point  by  noting  that  biases  by  those  making  the 
rating  calculations  and  inconsistency  in  data  accumulation  could  affect 
project  scores  so  that  projects  may  appear  more  or  less  promising  than 

they  really  are.  A  major  limitation  is  that  scoring  models  do  not  account 

1  ? 

for  project  interrelationships  with  respect  to  resource  utilization. 

Once  a  resource  constraint  becomes  binding,  the  portfolio  is  determined. 
Obviously,  there  could  be  slack  in  other  resources  and  a  different  port¬ 
folio  could  lead  to  a  more  optimal  allocation.  A  further  problem  occurs 
with  mutually  exclusive  versions  of  the  same  project  which  represent 
various  time  durations  depending  upon  the  level  of  resources  allocated. 
Scores  can  be  calculated  for  each  version  but  the  trade-off  between 

resource  levels  and  time  durations  makes  it  even  more  difficult  to  select 
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the  portfolio  with  the  optimum  allocation.  Also,  since  the  multiple 
criteria  ratings  are  aggregated  into  dimensionless  score,  there  can  be 
little  analysis  of  the  degree  to  which  the  resultant  portfolio  meets  the 
various  organizational  criteria.  In  fact,  it  could  be  possible,  although 
unlikely,  to  construct  a  portfolio  in  which  every  project  selected  scores 
low  on  a  critical  objective  but  very  high  on  all  other  criteria;  the  end 
result  being  a  portfolio  which  does  not  satisfy  the  most  important  objec¬ 
tive.  In  summary,  scoring  models  are  best  utilized  for  the  screening  of 
ideas  and  proposals  (boxes  5-8,  Figure  1-1)  but  not  for  the  selection  of 
projects  and  resource  allocation  decisions. 

Comparative  Methods 

Comparative  methods  such  as  Souder’s  Q-Sort  with  nominal  group 
interactive  processes, ^  and  paired  comparison  instruments  as  suggested 


k 


by  Souder^  and  Tauss,^ 


represent  another  class  of  qualitative  tech- 
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niques  in  which  respondents  are  required  to  successively  compare  a 
project  proposal  with  another  or  with  a  subset  of  proposals  and  specify 
which  is  preferred  with  respect  to  a  single  criterion.  A  preference 
rank  ordering  results  for  project  selection  and  development  of  the 
research  portfolio.  If  multiple  criteria  are  to  be  considered,  separate 
Q-Sorts  or  paired  comparison  experiments  are  conducted  for  each  criterion 
and  a  project  profile  is  developed  for  management  evaluation , ^  or  the 
sorter  is  forced  to  intuitively  incorporate  multiple  criteria  into  the 
selection  procedure  in  an  aggregate  manner. ^  In  using  these  techniques 
for  project  selection,  criteria  are  rated  by  assigning  number  values  on 
a  ratio  scale  through  pairwise  comparison  experiments.  Project  sub¬ 
scores  are  then  determined  by  a  pairwise  comparison  of  projects  for  each 
criterion.  A  final  project  score  is  determined  by  summing  the  products 
of  criterion  weights  and  project  subscores.  The  project  scores  can  then 
be  rank  ordered  for  project  selection. 

The  disadvantage  of  these  methods  is  similar  to  that  of  the  scoring 
models  with  respect  to  resource  allocation  and  analysis  of  goal  attain¬ 
ment  levels.  Another  disadvantage  of  the  comparative  approaches  occurs 
when  a  new  project  is  added  to  the  set  of  projects  considered  for  the 
portfolio.  In  this  case,  the  entire  exercise  must  be  repeated  since 
an  additional  project  will  affect  the  preference  rankings  of  other 
projects. ^  New  projects  do  not  require  a  repeat  of  the  selection  pro¬ 
cedure  when  scoring  models  are  used  as  project  scores  are  determined 
independently.  However,  repeating  any  algorithm  used  to  select  projects 
when  new  proposals  are  added  to  the  list  may  be  advantageous  when  the 
process  is  conducted  on  a  semicont inuous  basis,  especially  with  the 
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quantitative  models  that  also  address  the  resource  allocation  problem. 
The  main  contribution  of  comparative  techniques  to  the  total  R&D 
process  is  not  in  the  project  selection  area,  but  rather  in  the  develop¬ 
ment  of  multiple  criteria  and  associated  priority  rankings.  That  is, 
they  have  potential  for  applications  concerning  the  decision  processes 
represented  by  boxes  1  through  4  in  Figure  1-1. 

Economic  Index  Models 

Economic  or  profitability  index  models  represent  another  class  of 
techniques  introduced  in  the  early  1960,s  as  a  way  to  solve  the  project 
selection  problem.  These  models  are  used  to  generate  indices  of  the 
relative  worth  of  projects  based  on  economic  factors  usually  combined 
in  a  nonlinear  functional  form.  The  indices  are  then  rank  ordered  for 
selection  purposes  or  compared  against  some  criterion  such  as  a  minimum 

admissable  profit  index  to  determine  when  projects  should  be  rejected 
20 

or  terminated.  Models  of  this  type  include  project  index  functions 

21  2?  23  24 

developed  by  Pacifico,  Hart,  "  Villers,  and  Whaley  and  Williams; 
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Ansoff’s  profit  figure  of  merit;  Olsen’s  relative  value  index;  and 

27 

a  profit-cost  index  function  proposed  by  Hart.  These  models  are 

generally  extensions  of  simple  techniques  such  as  the  return  ratio  used 

28 

by  Bobis,  Cooke  and  Paden  in  which  the  project  index  is  constructed  as 
the  ratio  of  the  expected  future  returns  to  the  expected  future  research 
costs.  As  an  example  of  an  extended  technique,  Villers’  method  develops 
the  index  of  relative  worth  as  the  ratio  of  the  estimated  present  value 
of  future  earnings  of  a  project  minus  the  direct  costs  of  research  to 
the  total  estimated  cost  of  the  R&D  project  effort.  The  resulting  ratio 
is  converted  to  an  expected  value  by  multiplying  by  the  probabilties  of 
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research,  development,  process  and  commercial  successes.  The  selection 

procedure  requires  rank  ordering  of  the  project  indices  and  selecting 

projects  from  the  top  of  the  list  until  a  total  cost  constraint  prevents 
29 

further  selections.  In  addition  to  the  basic  index  methods,  other 

economic  models  develop  project  figures  of  merit  based  simply  on  the 

30 

internal  rate  of  return  or  project  net  present  value.  Souder  suggested 
a  more  sophisticated  technique  which  combines  a  Hertz  tv^e  risk  analysis 
model  with  Olsen’s  relative  value  index  method.  Still  other  economic 
models  use  the  project  index  or  figure  of  merit  as  objective  function 
coefficients  in  mathematical  programming  formulations  which  are  discussed 

in  the  next  section. 
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Although  Baker  reports  some  evidence  of  application  of  economic 
index  models,  there  are  limitations.  First,  the  models  assume  that 
project  selection  decisions  are  made  based  on  a  single  economic  criteria, 
disregarding  the  noneconomic  objectives  of  the  enterprise.  Second, 
constraints  other  than  the  research  budget  are  not  considered  in  port¬ 
folio  construction.  Third,  economic  indices  give  no  guidance  as  to 
which  version  of  a  project  should  be  selected  where  the  versions  repre¬ 
sent  varying  profiles  of  resource  allocation  yet  yield  similar  index 
33 

values.  In  summary,  economic  index  models  best  serve  for  the  screen¬ 
ing  of  project  proposals  or  for  the  termination  decision  (reference 
boxes  7,  8,  13,  and  16,  Figure  1-1)  but  not  for  the  project  selection 
and  resource  allocation  problem. 

Mathematical  Programming  Models 

Mathematical  programming  models  are  the  most  recent  class  of 
models  proposed  for  the  project  selection  problem  and  have  the  major 


benefit  of  addressing  the  resource  allocation  decision  through  equations 
that  constrain  an  objective  function  which  measures  the  benefit  contri¬ 
bution  of  a  selected  portfolio.  These  models  suggest  the  present  state- 
of-the-art  for  the  decision  process  represented  by  box  10,  Figure  1-1, 
and,  therefore,  a  more  thorough  review  of  them  will  be  given  here.  The 
first  linear  programming  model  formulation  for  R&D  project  selection  is 

o  A 

credited  to  Asher,  in  which  project  selection  in  a  pharmaceutical 

company  is  accomplished  by  maximizing  the  expected  discounted  net  present 

value  of  projects  subject  to  manpower  and  raw  material  availabilities. 

IS  lb 

Other  work  such  as  the  models  developed  by  Lockett  and  Gear,  Hamburg, 

37 

and  Bell,  incorporate  extensions  and  variations  to  Asher’s  formulation. 
Bell’s  model  which  includes  future  time  period  resource  constraints  is 
presented  to  illustrate  a  general  formulation: 


n 

Maximize  Z 

-  1 
i=l 

I  bijxij 
j=l 

(2.1) 

Subject  to: 

:  JL 

X-  •  <  1 

i  1 , . . . ,n.  , 

(2.2) 

n 
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J  =  1 

l 

i=l 

k  =  1,. .. ,N.  , 
P  = 

(2.3) 

xij  • 

>  0 

l  1 , . . . ,n.  , 

j  =  !»••*, m^ . 

Where:  b^j  is  the  value  of  version  j  of  project  i. 

m-^  is  the  number  of  alternate  versions  of 
project  i. 

n  is  the  number  of  projects,  both  on-going 


and  new. 
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aijkp  is  t^ie  amount  of  resource  type  k  planned 
(or  required)  for  version  j  of  project  i 
in  time  period  p. 

A^p  is  the  overall  availability  of  resource 
type  k  in  period  p. 

N  is  the  number  of  resource  categories 

considered, 

P  is  the  number  of  periods  in  the  planning 

horizon. 
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Integer  (zero-one)  programming  models  such  as  those  of  Freeman, 

39  40 

Beged-Dov,  and  Minkes  and  Samuel  were  introduced  to  prohibit  the 
inclusion  of  fractional  projects  in  the  final  portfolio.  That  is,  the 
xjj  project  version  variables  are  further  constrained  as  follows: 


Before  discussing  some  of  the  extensions  to  these  basic  formulations, 
it  is  important  to  comment  on  the  relative  merits  of  the  linear  and 
integer  programming  approaches.  Proponents  of  the  linear  formulation 
(where  may  take  on  any  value  in  the  continuum  0  5  x^j  -  1)  note 
that  existing  linear  programming  codes  efficiently  handle  large  scale 
problems  and  provide  useful  information  through  sensitivity  or  post¬ 
optimality  analysis.  However,  the  linear  program  results  may  include 
fractional  projects  which  enable  an  optimum  allocation  of  resources 
providing  that  the  divisability  of  projects  is  feasible  and  desirable, 
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but  could  lead  to  difficulties  in  actual  interpretation.  Asher 
suggested  that  any  fractional  projects  be  included  in  the  research  effort 
as  partial  versions  of  the  original  projects.  That  is,  some  work  should 
be  accomplished  on  the  project  in  the  current  period  rather  than  exclud¬ 
ing  the  project  entirely  and  letting  slack  exist  in  the  available  resources. 

/  o 

Lockett  and  Gearqj  suggest  that  fractional  projects  may  indicate  a  modi¬ 
fied  version  of  the  project  should  be  designed  which  would  be  of  smaller 
scope  than  the  original  project  version.  It  should  be  noted  that  in  both 
of  these  cases  the  fractional  projects  selected  for  research  are  not  the 
original  projects  provided  as  inputs  to  the  model,  but  rather  they 
actually  represent  project  versions  generated  by  the  solution  algorithm. 

In  another  study,  Lockett  and  Gear^  rounded  fractional  projects  to  the 
closest  integer  value  (zero  or  one)  for  development  of  the  portfolio, 
although  they  state  that  "the  approach  was  not  fruitful  in  finding  a 
stable  portfolio.”  While  interpretation  of  fractional  projects  is 
difficult,  rounding  to  integer  values  is  dangerous.  As  Wagner  notes, ^ 
a  rounded  solution  could  be  infeasible  or  it  could  be  feasible  but  not 
optimal.  Any  adjustment  to  the  fractional  solution  values  represents 
a  departure  from  the  original  problem.  On  the  other  hand,  integer 
formulations  are  solved  by  integer  programming  codes  that  restrict  the 

problem  size  and  any  sensitivity  analysis  must  be  accomplished  by 

46 

rerunning  the  models.  Another  disadvantage  occurs  with  integer 
programming  algorithms  in  those  instances  where  the  linear  solution 
would  result  in  fractional  projects.  In  these  situations,  the  integer 
solution  will  have  slack  in  one  or  more  of  the  absolute  resource  con¬ 
straints.  In  summary,  there  appears  to  be  a  trade-off  between 
fractional  projects  with  their  inherent  interpretat ional  difficulties 
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and  the  ability  to  handle  large  problems.  Integer  programming  formula¬ 
tions  represent  the  more  realistic  project  representation,  although  a 
means  of  improving  the  efficiency  of  associated  solution  algorithms 
should  be  developed.  In  this  regard,  Mandakovic^  has  recently  devel¬ 
oped  an  interactive  heuristic  procedure  for  a  decomposed  integer  pro¬ 
gram  formulation  which  is  capable  of  handling  very  large  size  problems 
and  is  a  promising  method  for  single  objective  project  selection  appli¬ 
cations. 

Many  extensions  have  been  made  to  the  mathematical  programming 

A  ft 

techniques  discussed  above.  Gear,  Lockett  and  Pearson4  provide  an 
excellent  review  and  assessment  of  nine  representative  formulations. 
Watters, ^  Brockhoff,^  and  Chames  and  Stedry^  incorporate  the  use  of 
probabilistic  constraint  rows  to  reflect  the  uncertainty  in  resource 
availability.  Hess,'’*'  Rosen  and  Souder,*^  and  Dean  and  Hauser^  include 
a  probability  of  success  factor  which  is  multiplied  times  the  project 
value  coefficient  in  the  objective  function  to  allow  for  technical  out¬ 
come  uncertainty,  and  Watters^  suggests  the  maximization  of  a  portfolio 

utility  function  to  account  for  uncertain  returns.  The  techniques  devel- 

56  57  58 

oped  by  Brandenburg  and  Stedrv,  Watters  and  Brockhoff  incorporate 

capital  budgeting  concepts  but  are  limited  in  that  constraints  other 

59 

than  the  budget  are  not  addressed.  The  techniques  of  Hess,  Rosen  and 
Souder,^  Brockhoff, ^  and  Bell*^  allow  for  the  inclusion  of  alternate 
versions  of  projects,  compulsory  projects,  and  project  interactions. 

The  models  proposed  by  Mandakovic^  and  Baker,  et.  al.,^  incorporate 
the  realistic  aspects  of  the  decentralized  structure  of  organizat ional 
R&D  decision  making  and  the  corresponding  hierarchies. 

The  major  disadvantages  of  most  of  the  quantitative  techniques 
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described  in  this  section  are  first*  a  failure  to  account  for  the  dynamic 
nature  of  R&D  projects,  and  second,  the  reduction  of  multiple  organiza¬ 
tional  goals  to  a  single,  typically  economic,  objective  for  opt imizat 1 on. 
These  two  aspects  of  the  decision  making  process  are  more  fully  described 
in  the  next  two  sections. 

Multistage  Project  Modeling 

Because  of  the  long-term  nature  of  R&D,  and  the  high  degree  of 

uncertainty  involved,  R&D  project  selection  and  resource  allocation 

models  should  account  for  the  multistage,  sequential  nature  of  projects 

and  explicitly  consider  the  uncertainty  that  exists  in  the  project 

evolution  and  resource  parameters  if  they  are  to  arrive  at  a  near 

optimal  project  selection  and  resource  allocation  decision. Dynamic 
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programming  formulations  (for  example,  Hess  and  Rosen  and  Souder  ; 
do  address  the  multistage  and  sequential  nature  of  many  R&D  projects. 
However,  these  models  cannot  efficiently  handle  multiple  resource  con- 

r  n 

straints  past  the  first  period.  An  alternate  method  of  accounting 

for  the  multistage  aspect  is  the  representation  of  R&D  projects  in  a 

decision  or  project  tree  format  that  is  used  to  develop  the  constraint 

equations  for  linear  or  integer  programming  algorithms.  The  major  work 

in  this  area  is  by  Gear  and  Lockett;^  Hespos;^  Gear,  Gillespie  and 
71  72 

Allen;  and  Flinn  and  Turbin.  Project  trees,  as  developed  by  Hespos, 
are  used  to  represent  the  evolution  of  projects  or  project  versions, 
resource  requirements,  and  any  associated  uncertainties  over  the  plan¬ 
ning  horizon,  thus  enabling  each  project  version  to  be  diagramat ical ly 
planned  on  a  common  time  scale.  As  part  of  the  research  reported  in 
Chapter  III,  project  tree  planning  is  described  in  conjunction  with 


i 


25 


the  multiple  objective,  zero-one  goal  programming  model,  presented. 

Multicriteria  Models 

None  of  the  models  surveyed  explicitly  address  a  multiple 

criteria  objective  function  with  noncommensurate  goals  for  the  dynamic 

case,  although  several  models  indirectly  approach  the  problem.  Dean 

7  3 

and  Roepcke’s  cost  effectiveness  model  includes  a  weighted  value 
coefficient  for  each  task  (project)  included  in  the  objective  function. 
The  values  are  determined  by  developing  a  list  of  independent  multiple 
objectives.  Tasks  are  rated  against  each  of  the  objectives  and  a 
final  weighted  value  is  determined  by  summation.  Nutt,  4  Adams,  and 
Chiu  and  Gear^  use  similar  concepts  in  arriving  at  dimensionless 
coefficients  for  projects  included  in  the  mathematical  programming 
objective  function.  In  these  approaches,  a  portfolio  of  projects  is 
developed,  however  it  is  not  possible  to  directly  determine  the  achieve¬ 
ment  level  of  the  various  objectives  through  an  analysis  of  model 
results.  Furthermore,  a  project  rated  high  on  lower  priority  objec¬ 
tives  but  low  on  the  highest  priority  goal  could  drive  this  project 
into  solution  while  failing  to  attain  the  highest  priority  goal. 

Rosen  and  Souder^  recognized  the  multiple  objective  nature  of 
the  project  selection  decision  and  considered  four  objectives  in  their 
model  by  using  an  objective  ordering  technique.  Projects  are  selected 
by  maximizing  expected  profits.  The  solution  is  then  checked  to 
determine  achievement  of  three  additional  objectives:  maximizing 
total  expected  output  (expected  research  successes) ,  achieving  a 
specified  return  on  expenditures,  and  achieving  a  predetermined  success 
level  for  a  specific  project.  If  any  of  these  additional  objectives 


26 


are  not  achieved,  the  problem  is  rerun  by  dropping  out  low  expected 

profit  projects,  relaxing  the  objectives,  or  changing  the  constraints. 

78 

Although  the  Chames  and  Stedry  model  was  concerned  with 

allocating  funds  over  various  research  areas  and  not  with  project 

selection,  it  is  reviewed  here  as  multiple  goals  are  considered. 

The  model,  which  is  based  on  chance-constrained  programming,  minimizes 

a  variety  of  expected  activity  costs  subject  to  ensuring  that  short- 

run  and  long-run  research  activity  levels  or  goals  are  met.  Chance 

constraints  are  used  to  ensure  that,  within  a  given  probability, 

resource  availabilities  are  not  exceeded  in  meeting  the  activity  levels. 

Utility  theory  has  also  been  suggested  as  a  means  of  incorporating 

multiple  objectives  (see  for  example  Watters^  and  Keefer^).  For 

the  selection  problem,  the  decision  maker’s  objective  is  to  maximize 

the  expected  utility  of  the  portfolio’s  multidimensional  return  function. 

The  method  of  determining  the  utility  function  is  extremely  difficult 

to  handle  from  a  practical  standpoint,  even  for  just  four  dimensions, 

since  subjective  questioning  is  required  to  discover  indifference  levels 

81 

between  alternative  options  presented  to  the  decision  maker.  In 

82 

addition,  Lee’s  research  noted  that  under  normal  circumstances  it  is 
very  difficult  for  decision  makers  to  measure  precisely  how  much  more 
important  one  goal  is  to  another  in  the  cardinal  values  used  in  utility 
theory . 

8  3 

In  a  recent  publication,  Muhlemann,  Lockett  and  Gear  extend 
previous  work  done  on  project  tree  and  mathematical  program  modeling 
to  the  multiple  objective  case.  While  their  method  is  applicable  for 
those  problems  in  which  the  goals  are  measured  in  common  dimensions, 
it  is  not  designed  for  the  more  realistic,  non commensurate  multicriteria 
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case.  Keown,  Taylor  and  Duncan  have  also  suggested  the  application  of 
integer  goal  programming  for  the  multiple  criteria  selection  problem. 
Although  their  technique  approporiately  addresses  noncoimnensurate  multi¬ 
ple  objectives  ranked  on  an  ordinal  scale,  it  is  limited  to  the  static 
case  and  to  small  scale  problems. 

Model  Prescriptions 

A  number  of  representative  models  have  been  identified  which 
possess  some  of  the  many  features  a  general  project  select  ion/resource 
allocation  model  should  have.  From  this  review  a  set  of  prescriptions 
can  be  developed  for  a  model  that  can  be  used  as  an  analytic  technique 
in  the  larger  R&D  decision  making  process.  In  summary  form,  the  project 
selection/resource  allocation  model  should  be  capable  of: 

1)  Selecting  a  set  of  research  and  development  projects  that 
are  chosen  to  maximize  the  attainment  of  multiple  objec- 
t  ives . 

2)  Incorporating  the  multistage  and  sequential  decision 
making  nature  of  R&D  projects. 

3)  Addressing  the  limitations  of  scarce  multiple  resource 
constraints. 

4)  Representing  mutually  exclusive  projects  or  versions. 

5)  Incorporating  discrete  probability  estimates  or  proba¬ 
bility  distributions  for  the  various  parameter  estimates, 
and  be  capable  of  considering  estimate  error. 

6)  Representing  the  interaction  and  interrelationships  of 
projects . 

7)  Representing  any  balance  required  in  the  R&D  program  such 


as  between  basic  research  and  development  work. 


8)  Considering  experience  and  knowledge  of  R&D  personnel  and 
interacting  with  these  personnel  in  the  larger  R&D  deci¬ 
sion  making  system. 

9)  Including  some  type  of  sensitivity  analysis  capability  to 
consider  the  nwhat  if"  type  questions  associated  with  the 
uncertain  R&D  selection/allocation  problem  environment. 

10)  Generating  solutions  to  realistic  size  problems. 

The  model  presented  in  the  following  chapter,  which  represents  the 
major  focus  of  this  research,  has  been  designed  with  these  prescript ions 
in  mind. 
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CHAPTER  III 


THE  MULTISTAGE  ZERO-ONE  GOAL  PROGRAMMING  MODEL 

This  chapter  presents  the  zero-one  goal  programming  R&D  project 
selection  model,  including  techniques  for  incorporating  the  multistage 
nature  of  R&D  projects  and  stochastic  parameters.  The  first  section 
includes  a  discussion  of  goal  programming  concepts,  formulations  and 
existing  solution  techniques.  Next,  project  decision  tree  planning  is 
described  as  a  means  of  representing  the  evolution  of  projects  over  the 
multiperiod  planning  horizon.  The  following  two  sections  incorporate 
project  tree  planning  techniques  into  goal  programming  model  formula¬ 
tions.  The  solution  algorithm  is  presented,  illustrated  and  tested  in 
the  final  sections  of  this  chapter. 

Goal  Programming  Concepts  and  Formulation 

The  R&D  project  selection  and  resource  allocation  decision  is 
made  in  an  environment  where  projects  should  be  selected  and  evaluated 
against  multiple  criteria  having  no  common  underlying  measure. ^  These 
multiple  criteria  are  not  necessarily  of  equal  importance  to  the  organi¬ 
zation  and  various  ranking  schemes  have  been  suggested  to  reflect  this 

2 

hierarchy  of  importance. 

The  discussion  in  the  preceding  chapter  indicated  a  number  of 
reasons  why  the  project  selection  techniques  which  have  heretofore  been 
discussed  in  the  literature  are  inadequate  in  addressing  the  full 
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complexities  of  the  selection  problem.  Hence,  another  approach  seems 

necessary.  Linear  goal  programming,  first  developed  by  Charnes  and 

Cooper^  and  later  refined  by  Ijiri,^  Lee,“*  and  Ignizio,^  provides  a 

partial  solution  to  some  of  the  deficiencies  noted  above;  namely  its 

ability  to  deal  with  problem  formulations  having  ordinally  ranked, 

noncommensurate  multiple  objectives.^  The  technique  has  been  applied  to 

related  decision  areas  such  as  selection  of  capital  investment  projects, 

9 

and  resource  allocation  in  nonprofit  institutions,  and  is  proposed  for 
the  R&D  project  selection/resource  allocation  problem. 

The  basic  concept  of  goal  programming  involves  the  incorporation  of 
the  multiple  objectives,  pertinent  tc  the  problem  being  considered,  into 
a  model  by  setting  quantified  levels  of  achievement  or  goals  for  each  of 
the  objec tives . ^  Primary  goals,  which  may  be  noncommensurate  in  their 
units  of  measurement,  are  ranked  on  an  ordinal  scale  to  reflect  their 
relative  importance  to  the  model  users.  Preemptive  priority  levels 
are  assigned  to  each  goal  to  preserve  the  ordinal  ranking,  thus  assur¬ 
ing  that  more  important  goals  are  achieved  before  attempts  are  made 
to  attain  goals  of  lesser  importance.  It  is  important  to  note  that 
goal  programming  also  allows  for  a  cardinal  ranking  of  primary  goals 
having  common  units  of  measurement  and  for  a  cardinal  ranking  of  any 
subgoals  of  a  primary  goal.  The  objective  in  goal  programming  is  to 
seek  out  that  solution  which  comes  as  close  as  possible  to  satisfying 
all  of  the  quantified  goals.  This  is  accomplished  by  minimizing  the 
over-  or  underattainment  of  the  goals  based  upon  their  relative 
importance.^  Over-  or  underattainment  of  goals  is  represented  by 
deviational  variables  in  system  and  goal  constraint  equations,  where 
system  constraints  define  the  relationships  among  the  decision  variables 
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and  goal  constraints  define  the  relationships  between  the  decision 
*  12 

variables  and  the  goals.  These  deviational  variables  are  expressed 

in  two  dimensions:  represents  positive  deviation  from  an  established 

goal  i  and  n^  represents  negative  deviation.  The  general  linear  goal 

13 

programming  model,  as  presented  by  Ignizio,  is: 

Find  x  =  )  so  as  to  minimize  (3.1) 

a  =  j  pk[?k(n,p]  .  (3.2) 

such  that: 

fAx)  +  ni  -  pi  =  bi  i  =  1 , . . .  ,m. ,  (3.3) 

and  x,  n,  p  >  0 

where:  x  is  the  vector  of  decision  variables 

(i.e.,  selected  projects). 

(3.2)  is  the  achievement  function  where  the 
dimension  of  a  represents  the  number  of 

preemptive  priorities  among  the  objectives. 

-  -  14 

is  the  priority  level  of  g^(n,p). 
g^(n,p)  is  a  linear  function  of  the  deviational 

variables,  which  may  be  cardinally  ranked. 

K  5  m  where  m  is  the  number  of  objectives. 

(3.3)  are  the  problem  objective  equations. 

f^  (x)  is  a  function  of  the  decision  variables 
associated  with  the  ith  constraint, 
b^  are  the  quantified  goals  for  the  objectives. 

In  formulating  the  project  selection  problem  as  a  linear  goal  program, 
the  decision  variables,  ,  representing  projects  to  be  selected  for 
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the  portfolio,  would  be  further  constrained  by  an  upper  bound  of  one  to 
preclude  duplicate  projects  being  accepted. 

Algorithms  for  solving  the  linear  goal  programming  model  include 
the  general  inverse  method  developed  by  Ijiri^  and  the  more  efficient 
modified  simplex  method  introduced  by  Lee.^  Research  endeavors  utili¬ 
zing  the  linear  goal  programming  formulations  and  solution  methods  must 
accept  the  divisability  requirements  placed  upon  the  decision  variables 
as  is  true  with  linear  programming.  In  the  previous  chapter  it  was 
pointed  out  that  linear  programming  algorithms  used  to  solve  the  project 
selection  problem  can  lead  to  portfolios  with  fractional  projects. 

Since  fractional  projects  lead  to  interpretation  difficulties,  and  since 
a  rounded-off  solution  is  generally  either  infeasible  or  not  optimal  for 
the  problem  being  solved,  an  integer  algorithm  is  necessary  to  meet  the 
zero-one  restrictions  on  the  variables. 

In  a  recent  Ph.D.  dissertation,  Morris^  adopted  cutting  plane, 
branch  and  bound  and  implicit  enumeration  techniques  in  developing 
algorithms  for  integer  goal  program  formulations.  The  implicit  enumera¬ 
tion  technique  was  specifically  designed  for  the  zero-one  problem  and 
is  based  on  the  additive  algorithm  of  Balas,^  as  modified  by  Glover. ^ 
The  technique  works  by  enumerating  all  project  selection  combinations 
represented  by  the  vector  x,  either  explicitly  or  implicitly.  Certain 
solutions  are  evaluated  and  then  logic  is  used  to  eliminate  further 
solutions  without  evaluating  them  explicitly*  Initial  solutions  are 
created  by  systematically  adding  free  variables  (variables  not  yet 

assigned  a  value  of  zero  or  one)  to  x  to  determine  if  goal  attainment 
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for  the  highest,  unachieved  priority  goal  can  be  improved.  When  no 
further  improvement  is  possible,  a  backtracking  routine  is  initiated 


by  removing  the  last  decision  variable  assigned  to  the  most  current 
solution.  From  this  modified  solution,  additional  solutions  are  created 
by  systematically  adding  remaining  free  variables  to  x  to  determine  if 
goal  attainment  levels  can  be  further  improved.  The  entire  process  is 
repeated  until  all  solution  combinations  have  been  explicitly  or  implic¬ 
itly  evaluated.  Even  though  the  algorithm  is  designed  to  minimize  the 
number  of  solutions  that  must  be  explicitly  evaluated,  large  scale 
problems  require  many  evaluations.  For  this  reason,  the  algorithm  is 
limited  to  problems  with  an  upper  bound  of  approximately  20  variables, 

50  constraints  and  10  priority  levels. 

Since  the  integer  goal  program  formulation,  with  the  associated 
implicit  enumeration  solution  technique,  addresses  an  important  aspect 
of  the  R&D  project  selection  problem,  the  existence  of  multiple  objec¬ 
tives  having  no  common  underlying  measure,  it  will  form  the  starting 
point  for  the  development  of  a  method  of  performing  R&D  project  selection 
in  realistic  situations.  However,  while  the  implicit  enumeration 
algorithm  is  limited  to  applications  involving  no  more  than  20  project 
proposals,  interviews  conducted  during  the  course  of  the  present  research 

indicated  that  practical  applications  involve  at  least  20  to  40  project 
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proposals  (see  also  Souder  and  Lockett  and  Gear  ).  For  instance, 
managers  of  the  research  and  development  laboratories  of  the  textile 
firm  and  the  phamaceutical  company  interviewed,  report  that  approximately 
30  to  40  ongoing  and  new  projects  comprise  the  set  of  projects  under 
evaluation  at  semi continuous  time  periods.  Therefore,  to  be  useful,  it 
is  necessary  to  design  an  algorithm  capable  of  solving  at  least  these 
medium  size  problems.  In  the  present  research,  a  zero-one  goal  program¬ 
ming  algorithm  is  developed  to  achieve  this  capability. 
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Zero-One  Goal  Programming  Algorithm 

The  principal  objective  of  this  research  is  to  develop  a  zero-one 
goal  programming  algorithm  for  solving  R&D  project  selection  problems 
of  realistic  size.  A  subgoal  of  this  objective  is  to  generate  good 
working  solutions  within  a  relatively  small  computation  time.  Since 
the  model  is  proposed  as  an  interactive  decision  aiding  tool,  it  is 
envisioned  that  model  users  would  benefit  from  having  a  range  of  solu¬ 
tions  for  qualitative  assessment.  Further,  if  stochastic  parameters  are 
considered  and  simulation  techniques  are  used  for  sampling  their  proba¬ 
bility  distributions,  many  model  runs  are  required  for  statistical 
hypothesis  testing.  Thus,  even  though  the  Morris  algorithm  provides 
exact  solutions  for  problems  with  up  to  20  variables,  the  relatively 
long  computation  time  for  upper  range  problems  makes  multiple  runs 
expensive.  For  example,  Morris  reported  that  a  19  variable  test 

problem  required  over  15  minutes  of  computation  time  on  an  IBM  370/158 
2  3 

system.  If  just  30  runs  were  desired  for  sensitivity  analysis  or 
hypothesis  testing,  the  cost  and  time  may  be  prohibitively  high.  The 
proposed  algorithm  is  based  on  heuristic  techniques  and  it  is  recognized 
that  one  of  the  disadvantages  of  heuristics  could  occur  during  the 
sensitivity  analysis  stage;  that  is,  a  range  of  solution  strategies 

close  to  the  exact  solution  may  be  influenced  by  both  the  changes  to 

24 

input  data  and  the  vagrancies  of  the  heuristic  procedure.  The  counter 
argument  for  heuristics  concerns  the  practical  aspect  of  obtaining  good 

solutions  to  medium  size  problems  unsolvable  with  existing  techniques, 

25 

and  with  the  further  advantages  of  speed  and  versatility. 

Before  describing  the  solution  algorithm,  the  following  terms 


are  defined: 
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=  The  set  of  indices  of  all  unassigned  or  free  project 
variables  ,  that  is,  those  not  yet  assigned  a  value 
of  zero  or  one, 

V9  =  The  set  of  indices  of  all  project  variables  in 

solution,  that  is,  those  x.  =  1. 

J 

V3  =  The  set  of  indices  of  all  project  variables  forced  out 
of  the  solution,  that  is,  those  =  0. 

U  =  The  set  of  goal  underachievement s  at  the  various 
priority  levels.  If  l\  is  attained,  l\  *  0. 

* 

lT  »  The  upper  bound,  or  best  solution  determined  so  far 

during  the  variable  selection  routine.  The  variables 

*  * 

in  V2  and  which  generate  U  are  recorded  as  V0  and 


kk 

U  =  The  upper  bound  or  best  solution  determined  so  far 


during  one  iteration  of  the  variable  exchange  routine. 

kk 


The  variables  in  d  associated  with  U  are 

★  k  kk 

recorded  as  Y0  and  . 

*  *  it  *  - 

U  =  The  minimum  or  best  solution  among  the  U  solutions 

determined  during  the  variable  exchange  routines.  The 
variables  in  and  associated  with  the  solution 

kkk  kkk  kkk 


U  are  recorded  as  and 

The  procedure  begins  with  a  one  pass  variable  selection  routine 
26 

adapted  from  Morris^  that  rapidly  builds  an  initial  solution.  During 

the  initialization  step,  all  x^  are  set  equal  to  zero  but  considered  as 

* 


free  variables  so  that  V2*V3*0,  and  U  is  computed  and  designated  as  U 
to  indicate  the  upper  bound  or  best  solution  so  far.  Next,  each  variable 
in  is  considered  individually  as  a  solution  variable  for  inclusion  in 
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V9  and  that  one  which  results  in  the  greatest  improvement  to  U  is 

,  * 
selected.  That  variable  is  removed  from  and  added  to  »  and  U  is 

changed.  This  process  continues  iteratively  until  no  further  reduction 

in  the  goal  underattainment  levels  is  possible.  It  is  noted  that  during 

•k 

this  routine,  if  a  variable  under  consideration  does  not  improve  U  ,  it 

£ 

is  added  to  V^.  When  V^=0,  the  procedure  terminates  and  U  with  the 

k  k  kkk  kkk  kkk 

associated  and  vectors  are  recorded.  Further,  U  ,  and 


are  initialized  at  U  ,  V9 ,  and  V^,  respectively,  and  saved  for  use  later 
on  in  the  algorithm. 

The  next  procedure  is  a  one-by-one  variable  exchange  routine  that 

k 

iteratively  operates  on  the  set  \?2 .  The  procedure  begins  by  transferring 

* 

all  variables  from  to  so  that  V^=0.  Then,  the  variable  with  the 

* 

smallest  index  is  removed  from  V ^  and  added  to  V^.  U  is  computed  and 

£  k 

recorded  as  U  and  the  variable  selection  routine  starts  again,  this 

kk 

time  attempting  to  find  solutions  that  improve  on  U  .  If  any  U  computed 

k 

during  this  stage  is  also  less  than  U  ,  the  solution  is  tested  against 

k  k  k  kkk 

U  to  determine  if  a  global  improvement  is  possible.  Tf  so,  U  and 

*  k k  kkk 

the  associated  and  vectors  are  changed  to  record  this  solution 

as  the  best  found  so  far.  The  procedure  continues  until  V^=0.  At  this 
time,  the  variable'  that  was  removed  from  is  replaced  in  and  the 
variable  having  the  next  largest  index  is  removed  for  the  next  iteration 
of  the  variable  selection  routine.  The  procedure  continues  until  all 

k 

variables  in  the  set  have  been  exchanged.  If  there  was  an  improvement 
to  U***  during  any  of  the  exchange  iterations,  the  entire  exchange  routine 

k  kkk  k  kkk  k  kkk 

starts  a>  aln  with  U  =U  ,  ,  and  .  The  procedure  terminates 

when  no  further  improvement  to  U  can  be  found.  A  f]ow  chart  illustra¬ 
ting  this  procedure  is  shown  in  Figure  3-1  on  pages  44  and  45. 


It  is  noted  that  the  exchange  routine  described  above  is  somewhat 
related  to  the  general  class  of  interchange  heuristics  employed  as  solu¬ 
tion  algorithms  for  2ero-one,  single  objective  programming  problems. 

2  7 

For  example,  in  the  warehouse  location  problem,  Kuehn  and  Hamburger 
use  a  "bump  and  shift”  heuristic  to  close  one  or  more  warehouses  that 
were  selected  to  be  opened  by  an  additive  main  program  heuristic  and  to 
open  other  warehouses  that  were  not  selected  for  opening  in  the  main 
program,  in  attempts  to  make  improvements  in  the  evaluation  criterion 
(minimize  distribution  system  costs).  For  another  example,  Cornuejols, 

OQ 

Fisher  and  Nemhauser  describe  a  pairwise  interchange  heuristic  for 
the  bank  account  location  problem  where,  given  an  initial  solution  of 
locations  for  bank  accounts,  attempts  to  improve  the  evaluation  crite¬ 
rion  (maximize  check  clearing  times)  are  made  by  exchanging  one  selected 
bank  location  for  another  not  yet  selected.  The  particular  entering¬ 
leaving  pair  is  selected  in  a  variety  of  ways,  for  example,  first 
improvement  or  maximum  improvement. 


Project  Trees 

Descriptive  studies  cited  in  Chapter  II  note  that  to  be  useful, 

R&D  project  selection  and  resource  allocation  models  should  account  for 
the  multistage,  sequential  nature  of  projects  and  the  uncertainty  that 
exists  in  project  evolution  and  resource  parameters.  As  is,  the  Morris 
integer  goal  programming  formulation  does  not  do  this.  The  use  of 
project  trees  in  combination  with  a  mathematical  programming  technique 
and  simulation  or  with  stochastic  programming  techniques  are  methods  of 
incorporating  this  dynamic  nature  and  uncertainty  into  the  goal  program¬ 
ming  format,  and  this  further  improve  our  ability  to  address  the 


Figure  3-1 

Solution  Procedure  Flow  Chart 


inadequacies  of  the  model  discussed  in  the  review  of  the  literature 
(Chapter  II). 

The  project  tree  structure  is  a  promising  means  of  analysis  as  it 
can  be  used  to  represent  many  of  the  aspects  of  the  R&D  project  selection 
environment.  In  addition  to  incorporating  the  multistage  nature  of  R&D 
projects,  the  representation  allows  for: 

1)  The  examination  of  the  relative  merits  of  series  or  parallel 
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strategies  (see  for  example,  Abernathy  and  Rosenbloom  ). 

2)  The  analysis  of  various  start  delays  and  rates  of  resource 
usage* 

3)  The  representation  of  uncertaint ies  in  project  duration, 
resource  requirements,  project  outcomes  and  project  values. 

4)  The  inclusion  of  a  mixture  of  applied  research,  basic 
research  and  development  projects. 

5)  The  representation  of  mutually  exclusive  or  dependent  projects. 

The  project  trees  in  Figure  3-2  follow  the  format  presented  by 
30 

Gear  and  Lockett  and  are  included  to  illustrate  some  of  the  features 
listed  above*  In  this  figure,  projects  1  and  2  have  deterministic 
technical  outcome  paths  but  reflect  different  resource  consumption 
patterns,  time  durations  and  expected  outcome  values.  Project  3  can  be 
performed  in  either  of  two  mutually  exclusive  versions,  x^  or  x^ ^ »  the 
second  of  which  has  a  one  year  start  delay  and  a  higher  expected  value 
when  compared  to  the  first  version.  Project  4  has  a  period  of  research 
followed  by  a  chance  node  reflecting  an  uncertain  technical  outcome  with 
two  future  events  that  would  dictate  different  funding  levels  in  the 
second  year  and  result  in  different  expected  values  depending  upon 


Figure  3-2 

Illustrative  Project  Trees 
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whether  chance  branch  A  or  chance  branch  B  occurs.  The  resource  alloca¬ 
tion  problem  is  thus  stated  as:  what  subset  of  projects  should  resources 
be  allocated  to  in  order  to  be  on  an  optimal  path  in  terms  of  maximizing 
the  sum  of  terminal  values  for  projects  eventually  completed? 

By  using  project  trees  in  the  manner  described,  the  multiperiod 
nature  of  R&D  projects  can  be  explicitly  recognized  and  appreciated  in 
the  project  selection  decision. 

Consideration  of  the  Stochastic  Nature  of  R&D  Project  Selection 

The  high  risk  and  uncertainty  which  characterize  R&D  limit  the 
utility  of  project  selection  models  which  are  confined  to  deterministic 
representations  of  the  problem.  Accordingly,  simulation  or  stochastic 
programming  has  been  suggested  to  incorporate  the  stochastic  parameters 

and  chance  future  events  from  the  project  tree  diagram  for  the  single 
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objective  problem.  This  research  extends  these  techniques  to  the 
multiple  noncommensurate  objective  case  with  discrete  project  variables. 

To  illustrate  the  stochastic  formulation  of  the  integer  goal  pro¬ 
gram,  the  R&D  project  selection  problem  already  presented  in  the  Figure 
3-2  project  tree  description  will  be  used  for  an  example.  Let  the  first 
priority  goal  be  to  require  the.  inclusion  of  project  3  in  the  research 
portfolio.  This  goal  is  representative  of  realistic  situations  in  which 
the  R&D  manager  must  include  specifically  identified  project  proposals  in 
the  research  effort,  where  such  identification  normally  comes  from  higher 
organizational  levels.  For  example,  one  proposal  mav  represent  a  defense 
project  which,  by  itself,  has  a  low  return  to  the  firm  and  would  not 

normally  be  selected,  but  is  expected  to  generate  a  large  follow-on 
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contract  business.  In  this  case,  top  management  may  intervene 


49 


in  the  selection  process  and  request  this  project  be  funded.  The  second 

priority  goal  is  not  to  exceed  the  budget  levels  in  each  of  the  three 

time  periods,  and  the  third  priority  goal  is  to  maximize  the  expected 

value  of  the  portfolio  selected. 

The  constraint  equations  for  the  stochastic  goal  programming 

formulation  are  shown  in  Table  3-1.  The  first  goal  is  achieved  by 
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minimizing  n^  and  p^  in  rows  1  and  2  of  the  table.  The  second  goal 
is  satisfied  by  minimizing  the  positive  deviational  variables  in  rows 

3  through  6.  Row  3  represents  the  first  period  budget  constraint  goal; 
rows  4  and  5  represent  the  second  neriod  budget  goal,  one  row  for  each 
possible  future  outcome  of  project  4;  and  row  6  represents  this  goal  in 
the  third  period.  The  third  goal  is  achieved  by  minimizing  n_,  from  a 
large,  unattainable  right  hand  side  value.  All  technological  coeffi¬ 
cients  in  the  constraint  equations  are  obtained  directly  from  Figure  3-2 
except  the  expected  value  for  project  4.  The  expected  value  of  project 

4  is  the  probability  of  chance  branch  A  times  the  expected  value  of  x 

4A 

plus  the  probability  of  chance  branch  B  times  the  expected  value  of  x/t5: 

4D 


Expected  Value 
Project  4 


=  (.3)(5.4)  +  ( . 7) (5 . 0)  =  5.12 


(3.4) 


Table  3-1 

Model  Constraints  for  Example  Problem 

Variables 


Row 

No. 

X1 

x2 

X31 

X 

OJ 

ro 

X4 

ni 

Pi 

Sign 

RHS 

1 

1 

1 

ni 

"pl 

= 

1 

2 

1 

1 

n2 

~P2 

= 

1 

3a 

2 

2 

3 

1 

°3 

"P3 

= 

10 

4a 

3 

2 

3 

3 

3 

n4 

"P4 

= 

10 

5a 

3 

2 

3 

3 

2 

n5 

"P5 

= 

10 

6a 

3 

3 

n6 

“P6 

= 

10 

7b 

5 

2.8 

4 

6 

5.12 

n7 

~P7 

= 

25 

a(105  $);  b(106  $) 
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The  integer  goal  programming  formulation  also  requires  Xj-0,1  and 
n^,  p^-0  for  all  j  project  variables  and  i  deviational  variables. 
The  objective  function  is; 


Minimize  “ 

cl 


Pl(nl+P2);  P2(p3+P4+P5+P6) 5  P3(n7) 


(3.5) 


The  disadvantage  of  the  stochastic  programming  approach  described 

above  is  that  with  many  chance  future  events,  the  constraint  set 
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becomes  unmanageable.  For  example,  Lockett  and  Gear  examined  an 
industrial  case  with  37  projects,  65  decision  nodes  and  40  chance  nodes 
involving  four  time  periods  and  six  resource  categories.  They  reported 
that  the  size  of  the  problem  prohibited  the  application  of  stochastic 
linear  programming  as  the  number  of  constraints  ran  into  millions. 


Obviously,  this  is  also  a  disadvantage  of  the  stochastic  zero-one 
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goal  programming  formulation.  Another  disadvantage  of  the  stochastic 
linear  or  goal  programming  approaches  is  that  the  solution  represents 
the  best  portfolio  strategy  for  all  possible  future  events,  without 
recognizing  that  the  occurrence  of  some  of  these  events  could  be  highly 
unlikely. 

One  alternative  solution  technique  is  simulation  where  each  chance 

node  is  sampled  by  Monte  Carlo  processes  to  determine  which  path  and 

associated  path  variable  from  the  project  trees  is  to  be  input  into  the 
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mathematical  program.  Chance  nodes  are  repeatedly  sampled  and  the 
results  of  associated  mathematical  programs  are  recorded.  In  a  large 
problem,  it  can  be  expected  that  many  portfolios  will  be  generated  for 
the  simulated  possible  outcomes.  These  portfolios  can  then  be  subjected 
to  statistical  analysis  in  order  to  compare  the  stability,  objective 
values  and  frequency  of  portfolio  occurrences. 

Another  simulation  technique,  which  utilizes  the  model  as  a  truely 
effective  interactive  device,  is  suggested  when  several  of  the  many 
chance  future  events  are  clearly  dominant  or  otherwise  of  interest  to 
the  decision  makers.  In  this  case,  the  model  users  would  predetermine 
the  occurrence  of  these  future  events  and  provide  the  associated  path 
variables  as  input  data  for  the  model;  and  receive,  as  output,  the 
suggested  portfolio  strategy  and  associated  goal  attainment  levels  for 
further  statistical  or  subjective  analysis.  In  this  manner,  the  model 
users  are  able  to  evaluate  strategies  associated  with  specific  or 
dominant  future  events  in  contrast  with  the  stochastic  approach  which 
recognizes  the  existence  of  all  future  events,  including  those  that  are 


highly  unlikely. 


Table  3-2 

Model  Constraints  for  Example  Problem 


Sign  RHS 

=  1 

=  1 

=  10 

=  10 

=  10 

=  25 


To  illustrate  the  zero-one  goal  programming  formulation  with 
simulation  technique,  the  example  problem  described  in  Figure  3-2 
will  be  used.  For  this  illustration,  the  model  users  are  interested 
in  the  portfolio  strategy  suggested  if  dominant  branch  B  of  the  project 
tree  shown  in  Figure  3-2  occurs.  The  model  constraints  are  provided 
in  Table  3-2  above  and  the  objective  function  is: 

Minimize  *  = 

Solutions  to  this  and  the  stochastic  zero-one  goal  programming 
formulations  will  be  presented  in  the  next  section. 


Pl(nl+P2);  P2(P3+P4+P5);  P3(n6) 


(3.6) 


Variables 


Row 

No. 

X1 

X2 

X31 

X32 

X4B 

ni 

?i 

1 

1 

1 

ni 

"P1 

2 

1 

1 

n2 

"p2 

3a 

2 

2 

3 

1 

n3 

~P3 

4a 

3 

2 

3 

3 

2 

n4 

"P4 

5a 

3 

3 

n5 

~p5 

6b 

5 

2.8 

4 

6 

5 

n6 

'P6 

a(105  $);  b(106  $) 
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An  Illustration  of  the  Solution  Algorithm 

To  illustrate  the  variable  selection  and  exchange  heuristic 
proced  3,  the  example  R&D  project  selection  problem  shown  in  the 
project  trees  of  Figure  3-2  is  presented.  The  stochastic  zero-one  goal 
programming  formulation  previously  described  and  shown  in  Table  3-1  will 
be  used  in  this  illustration. 

Step  1:  The  first  step  is  to  set  all  variables  equal  to  zero 
and  obtain  an  initial  solution: 

Vj*  (1,2,31,32,4) 
v2=  0 
V3=  0 

U*  =  U  =  (1,0,25) 

Step  2 :  Next,  that  variable  which  will  improve  U*  to  the 
greatest  extent  is  selected  for  solution.  Underachievement  exists 
at  the  first  priority  level,  and  involves  the  negative  deviational 
variable  n^.  Consequently,  the  variable  with  the  largest  positive 
coefficient  in  row  1  is  chosen  for  solution.  In  this  case,  there  is 
a  tie  between  x  and  x  which  is  broken  by  arbitrarily  selecting  the 
variable  with  the  smallest  index  number,  x^.  In  the  computer  program, 
if  one  priority  level  has  more  than  one  row  with  negative  deviational 
variables,  the  sum  of  the  coefficients  over  these  rows  is  computed  for 
each  respective  variable  to  determine  the  largest  sum.  It  can  be  seen, 
then,  that  to  minimize  goals  of  negative  deviations,  variables  are 
selected  that  have  large  coefficients.  The  results  at  the  end  of  this 
step  are: 

Vx«  (1,2,32,4) 


V2*  (31) 
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V3=  0 

U*  =  U  =  (0,0,21) 

Step  3;  The  first  two  goals  are  completely  satisfied  but  the  fhird 
is  underattained.  The  third  goal  involves  minimization  of  the  negative 
deviation  in  row  7  so  the  variable  from  with  the  largest  coefficient 
in  row  7  is  added  to  V^,  provided  that  this  selection  does  not  result 
in  an  increase  in  the  underattainment  of  the  higher  priority  goals. 

For  example,  x^  has  the  largest  coefficient  of  the  variables  in  row  7 
of  Table  3-1,  but  its  selection  would  result  in  U  =  (1,0,13)  which  is 
not  less  than  11*.  Therefore,  V^=  (32)  and  the  variable  with  the  next 
largest  coefficient  is  examined.  The  results  at  the  end  of  this  step 
are  : 

v  (1,2) 

V2-  (31,4) 

V3*  (32) 

U*  -  U  -  (0,0,15.88) 

Step  4? 

Vj=  (2) 

V2=  (31,4,1) 

V3=  (32) 

U*  =  U  =  (0,0,10.88) 

Step  5: 

V  -  0 

V2»  (31,4,1) 

V3«  (32,2) 

U*  *  U  *  (0,0,10.88) 

Since  Vj*®,  the  variable  selection  routine  is  completed  and 
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***  ***  *** 

U  =  (0,0,10.88),  V2  =  (31, A, 1),  and  =  (32,2).  The  exchange 

procedure  is  not  initiated. 

Step  6;  (Exchange  variable  x^) 

Vj-  (32,2) 

V2=  (31,4) 

v3=  (1) 

V**  =U=  (0,0,15.88) 

Step  7 :  (Begin  the  variable  selection  routine) 

V2=  (32) 

V2=  (31,4,2) 

V3=  (1) 

U**  =  U  =  (0,0,13.08) 

Step  8: 

v2=  0 

V2=  (31,4,2) 

V3=  (1,32) 

hL  JL 

U  =  U  =  (0,0,13.08)  (End  the  variable  selection  routine) 

Step  9:  (Exchange  variable  x.^) 

Vj=  (32,2) 

V2=  (4,1) 

V3=  (31) 

U**  =  U  =  (1,0,14.88) 

Step  10:  (Begin  the  variable  selection  routine) 

Vj=  (2) 

V2=  (4,1,32) 


V3»  (31) 
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U  =  (0,0,8.83).  Since  U  is  less  than  U* ,  U  ,  and  U  : 


U***= 

(0,0,8.38) 

r**_ 

*** 

(A, 1,32) 

2  ~ 

V2  = 

** 

*** 

(31) 

V,  = 

Step  11; 

Vj®  0 

V2=  (4,1,32) 

V3=  (2,31) 

U  =  (0,0,8.88)  (End  the  variable  selection  routine.) 

The  variable  exchange  routine  is  repeated  once  more,  with  forced 
out  of  the  solution,  thus  completing  the  exchange  of  all  the  variables 
in  at  the  end  of  step  5.  The  best  solution  during  the  three 
exchange  iterations  was  determined  to  be  U  =  (0,0,8.88)  and  V  = 
(1,32,4).  The  variable  exchange  heuristic  is  then  repeated  with  this 
solution  set  and  no  further  improvement  was  found  in  the  nine  addi¬ 
tional  steps.  Note  that  while  the  problem  was  solved  in  20  steps,  some 
of  the  steps  are  for  record  keeping  and  the  actual  number  of  iterations 
involving  the  evaluation  of  solution  combinations  was  10,  which  is 
about  one  third  of  the  possible  32  solution  combinations  (2^).  The  best 
solution,  which  is  also  optimal,  then  is  to  select  projects  1  and  4  and 
version  2  of  project  3.  The  first  two  goals  are  attained  and  the 
expected  benefit  for  the  third  goal  is  $  16,120,000  ($  25,000,000  - 
$  8,880,000). 

In  conclusion,  it  is  important  to  note  that  the  stochastic  formu¬ 
lation  and  resultant  solution  provides  the  optimal  strategy  for  both 
possible  future  events,  without  recognizing  that  the  occurrence  of 
event  A  is  unlikely.  If  the  problem  is  resolved  considering  only  the 
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dominant  future  event  B,  the  solution  is  to  select  projects  1,  2, 
version  2  of  project  3,  and  project  4  with  a  resultant  expected  benefit 
of  $  18,800,000.  This  solution  is  for  the  zero-one  goal  programming 
formulation  with  simulation  techniques  as  was  described  in  Table  3-2. 
Dominant  future  event  analysis  is  discussed  in  greater  detail  in 
Chapter  IV. 

Results  of  Testing  the  Heuristics  Against  the  Morris  Algorithm 

The  variable  selection  and  exchange  heuristic  procedures  pre¬ 
sented  here  were  tested  on  six  problems  which  are  described  in  Table 

3/ 

3-3.  Problems  1  and  4  through  6  are  from  Morris  and  problems  2  and 
3  were  created  to  test  additional  problem  structures.  Computer 
program  runs  were  performed  on  an  IBM  370/155  system,  except  where 
indicated,  and  CPU  time  reported  is  for  computation  and  printing  the 
final  solution.  The  results  for  the  one-bv-one  exchange  heuristic 
reflect  substantial  time  reductions  while  reaching  the  optimal  solutions 
for  four  of  the  six  problems.  Analysis  of  the  two  solutions  where 
optimality  was  not  achieved  indicate  that  the  procedure  was  locked  into 
local  minima  on  the  multidimensional  goal  attainment  response  surface. 
Accordingly,  a  partition  procedure  was  added  to  the  algorithm  so  that 
several  forced  subproblems  are  created  and  solved.  The  solution  with 
the  best  goal  attainment  levels  (Minimum  U  )  is  selected  as  the 
solution  to  the  original  problem.  The  partition  procedure  was  designed 
for  the  particular  structure  of  the  R&D  project  selection  problem  where 
it  is  often  the  case  that  one  or  more  projects  are  proposed  in  mutually 
exclusive  project  versions,  thereby  generating  system  constraints  of 
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the  type  illustrated  below: 


'll 


+  x 


12 


<  1. 


(3.7) 


This  equation  is  transformed  into  a  goal  programming  format  as 
follows : 


X11  +  Xl')  +  nl  ~  P1  =  (3.8) 

and  the  objective  is  to  minimize  p^  at  the  first  priority  level  which 
guarantees  solution  feasibility.  For  this  equation,  two  artificial 
problems  can  be  created;  one  with  x  forced  out  of  the  solution  and 
the  other  with  x  forced  out.  The  procedure  is  performed  by  holding 
the  selected  variable  in  the  set  of  projects  forced  out  of  the  solution 
(V^)  during  the  variable  selection  and  exchange  operations. 

While  the  partition  technique  was  designed  for  the  particular 
structure  of  the  R&D  project  selection  problem,  the  procedure  is 
general  and  may  be  applied  to  problems  not  having  mutually  exclusive 
system  constraints.  For  medium  size  problems  with  up  to  45  project 
variables,  full  partitioning  may  be  efficiently  employed  by  creating 
one  subproblem  for  each  project  variable.  For  larger  problems, 
selective  partitioning  may  be  effectively  used  by  first  running  the 
model  with  only  the  variable  selection  and  exchange  heuristics  and 
then  rerunning  the  model  by  partitioning  on  the  variables  in  solution. 

The  model  results  for  the  partition  procedure  combined  with  the 
variable  selection  and  exchange  heuristics  are  also  shown  in  Table  3-3. 
For  problems  1,  2,  3,  5  and  6,  partitioning  was  performed  on  the 
mutually  exclusive  project  versions.  For  problem  4,  full  partitioning 
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was  used  due  to  the  absence  of  mutually  exclusive  system  constraints. 

As  these  results  indicate,  improved  solutions  may  be  obtained  by  using 
the  partitioning  procedure.  The  reason  for  this  is  due  to  the  operation 
of  the  exchange  procedure,  which  interchanges  one  solution  variable  with 
one  or  more  variables  not  yet  in  the  solution  at  each  iteration.  Other 
procedures  such  as  exchanging  two  or  three  solution  variables  with  one 
or  more  variables  not  yet  in  the  solution  could  increase  the  chance  of 
reaching  the  global  minimum  but  only  at  the  expense  of  increased  model 
complexity  and  computation  time.  The  partition  procedure  is  a  simple 
alternative  and  is  shown  to  be  efficient  in  tests  of  the  algorithm. 

A  user’s  guide  for  the  associated  computer  program  is  provided 
38 

in  Appendix  A.  This  guide  includes  a  description  of  the  computer 
code,  data  card  preparation  instructions,  and  suggestions  for  using 
the  three  options  of  the  code  which  are  summarized  as  follows: 

1)  Option  1:  This  option  employs  the  variable  selection 
heuristic  without  the  exchange  procedure. 

2)  Option  2:  This  option  incorporates  both  the  variable 
selection  and  exchange  procedure  as  previously  described 
in  this  chapter. 

3)  Partitioning  Option:  The  partitioning  procedure  may  be 
used  with  either  option  1  or  2  above. 


The  computer  code  with  sample  output  is  provided  in  Appendix  B. 
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CHAPTER  IV 


R&D  PROJECT  SELECTION  CASE  STUDY 

This  chapter  presents  a  hypothetical  R&D  project  selection  case 
study  to  provide  a  comprehensive  demonstration  of  the  zero-one  goal 
programming  model.  The  project  selection  problem  is  first  described 
in  the  project  tree  format  discussed  in  Chapter  III.  Methods  for 
modeling  the  more  complex  aspects  of  parallel  versus  series  research 
strategies  and  dependent  project  interrelationships  are  included. 

Next,  organizational  objectives  are  developed  and  the  goal  programming 
model  is  formulated  using  information  from  the  project  trees.  Two 
experiments  are  then  described,  the  first  of  which  concerns  analysis 
of  research  strategies  associated  with  various  future  technical  out¬ 
come  events  of  interest  to  the  model  users.  The  second  experiment 
involves  sensitivity  analysis  on  the  multiple  objective  priority 
structure.  The  chapter  concludes  with  a  discussion  of  an  alternate 
method  of  using  the  model  for  determination  of  the  research  portfolio. 

Problem  Description  and  Presentation  of  the  Project  Trees 

An  industrial  R&D  laboratory  must  select  a  portfolio  from  a  set 
of  proposals  for  projects  to  be  performed  over  a  three  year  planning 
horizon.  There  are  two  resource  categories  and  a  total  of  ten  projects 
several  of  which  have  multiple  versions.  Each  of  these  projects,  along 
with  their  yearly  budget  and  scientific  staff  resource  requirements,  is 
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represented  in  the  project  trees  diagrammed  in  Figure  4-1.  The  path 
variable  notation  in  the  righthand  column  is  included  for  project  iden¬ 
tification  purposes.  There  are  nine  decision  nodes  and  six  chance 
nodes,  the  latter  involving  14  technological  outcome  events  identified 
by  the  branches  and  probabilities  labelled  A  through  N.  The  probabil¬ 
ity  of  occurrence  of  any  future  event  in  combination  with  other  future 
events  will  be  referred  to  as  a  future  technological  outcome  state,  or 
simply,  the  future  state.  When  discussed,  a  future  state  will  be  iden¬ 
tified  by  the  labels  of  its  component  event  branches;  thus,  BCFHIL 
represents  one  of  the  96  future  states  that  could  occur  over  the  three 
year  planning  horizon.  It  can  be  observed  that  some  of  the  future 
states  are  more  dominant  than  others.  For  example,  future  state  BCFHIL 
has  a  probability  of  occurrence  of  .09  which  is  over  500  times  more 
likely  than  the  future  state  ACEKJK  which  has  a  probability  of  occur¬ 
rence  of  less  than  .0002. 

Projects  1  through  5  and  7  through  9  are  representative  of  model¬ 
ing  techniques  already  described  in  Chapter  III  and  only  a  brief 
description  is  necessary  here.  Projects  1  through  4  have  deterministic 
resource  requirements  and  technological  outcomes.  Project  5  consists 
of  two  mutually  exclusive  project  versions  with  version  1  having  a  one 
year  start  delay.  Project  7  has  an  initial  year  of  research  after  which 
uncertainty  exists  as  to  the  level  of  resource  usage  in  the  second  year 
required  to  achieve  the  expected  benefits.  Project  8  also  has  an  ini¬ 
tial  year  of  research  after  which  the  uncertain  technological  outcome, 
represented  by  the  chance  node,  reflects  project  continuation  or  termi¬ 
nation.  Project  9  has  two  periods  of  development  work  followed 
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by  the  uncertain  outcome  of  either  additional  work  required  or  early 
completion.  Projects  6  and  10  represent  modeling  extensions  discussed 
in  the  next  subsections. 


Project  Tree  Modeling  of  Mutually  Dependent  Projects 

Project  6  is  used  to  illustrate  the  method  suggested  for  modeling 
dependent  project  interrelationships.  This  project  actually  represents 
two  distinct  projects,  one  of  which  is  shown  as  path  variable  7  in 
Figure  4-1,  and  hereafter  referred  to  as  project  6-1.  The  other  project 
is  not  included  directly  in  the  figure  and  will  be  referred  to  as  pro¬ 
ject  Z,  Project  Z  requires  spending  levels  of  S  300,000  in  year  1  and 
$  100,000  in  year  2,  and  2  scientists  in  years  1  and  2.  The  antici¬ 
pated  benefit  of  project  Z  is  $  15,000,000  and  it  is  expected  to 
increase  the  firm's  market  share  by  .5  percent.  Commercially,  the 
projects  are  mutually  dependent  and,  technically,  if  Z  is  selected, 
project  6-1  must  also  be  included  in  the  portfolio  although  project 
6-1  may  be  selected  by  itself.  The  probability  of  commercial  success 
of  project  Z  is  .40;  and  this  probability,  given  that  project  6-1  is 
successful,  increases  to  .80.  The  probability  of  project  6-1  being 
successful  given  that  Z  succeeds,  is  1.00.  With  this  information,  a 
new  project  version,  noted  as  path  variable  8  is  generated  and  shown 
in  Figure  4-1.  This  version  represents  the  resource  requirements  for 
both  projects  Z  and  6-1  and  the  combined  benefits  of  each.  The  prob¬ 
ability  of  commercial  success  is  the  probability  of  both  projects  Z 


and  6-1  being  successful: 


p(Zn6-l)  =  p(Z)p(6-l | Z)  =  p(6-l)p(Z | 6-1)  =  .40 


(4.1) 


Following  this  method,  the  final  decision  tree  for  project  6  now 
represents  two  mutually  exclusive  project  versions.  If  the  version 
represented  by  path  variable  7  is  selected,  only  project  6-1  will  be 
funded.  If  path  variable  8  is  selected,  then  both  projects  Z  and  6-1 
will  be  funded.  Of  course,  neither  version  may  be  selected  in  the  final 
port f olio. 

Project  Tree  Modeling  of  Parallel  Versus  Series  Strategies 

When  there  exists  more  than  one  approach  for  conducting  a  research 

project,  Abernathy  and  Rosenbloom^  suggest  that  decision  makers  should 

examine  the  relative  merits  of  a  parallel  strategy  before  adopting  the 

normally  used  series  strategy.  The  strategy  of  employing,  in  parallel, 

several  research  or  development  approaches  to  the  same  objective  has 

also  been  suggested  elsewhere,  especially  in  military  development 

2 

policy.  For  instance,  Mansfield* s  research  found  that  in  weapons 

3 

systems  development  programs.  Burton  Klein  of  the  Rand  Corporation 
suggested  parallel  development  efforts  be  used  to  overcome  difficult 
technological  problems;  and  Peck  and  Scherer  argued  for  parallel 
approaches  when  the  weapons  development  program  represents  a  major 

A 

state-of-the-art  advance  and  minimizing  development  time  is  cri' ical. 

The  series  strategy  involves  selecting  one  of  the  multiple 
approaches  for  research.  In  the  case  of  technical  outcome  failure, 
another  approach  is  then  initiated  and  so  on,  until  success  occurs  or 
all  of  the  alternative  approaches  have  failed.  The  other  strategy 
involves  funding  all  approaches  in  parallel  and  continuing  until  a 
clear  choice  can  be  made  towards  the  objectives.  Note  that  parallel 
approaches  are  not  to  be  confused  as  duplicate  projects  since  they 
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involve  distinctly  different  means  to  the  same  end  objective.  Gener¬ 
ally,  a  series  strategy  results  in  a  longer  commitment  of  resources 
whereas  the  parallel  strategy  requires  greater  resource  commitment  in 
the  initial  periods.  Project  10  is  included  to  illustrate  the  model¬ 
ing  techniques  suggested  to  incorporate  these  strategies  into  the 
project  tree  format. 

Project  10  represents  a  fixed  fee  contract  for  a  process  modifi¬ 
cation  that,  if  technically  successful,  would  result  in  an  estimated 
$  7,000,000  reward  in  follow-on  contracts  for  the  firm.^  Two  alterna¬ 
tive  approaches,  which  do  not  appear  directly  in  Figure  4-1,  exist  for 
conducting  the  development  work.  The  first  approach,  hereafter 
referred  to  as  approach  10A,  requires  funds  of  $  200,000  and  3  scien¬ 
tists  in  the  first  two  years  of  development  work,  and  the  second,  here¬ 
after  referred  to  as  approach  10B,  requires  $  300,000  and  4  scientists 
in  the  first  two  years.  It  is  assumed  that  after  the  first  year  of 
development,  the  progress  made  on  either  approach  will  be  sufficient  to 
indicate  project  success  or  failure.  After  the  initial  year  of  work  is 
conducted,  10A  has  a  probability  of  technical  success  of  .40  and  10B 
has  a  probability  of  technical  success  of  .80.  The  decision  to  be  made 
is,  if  project  10  is  selected,  should  a  parallel  or  series  strategy  be 
adopted  for  the  two  available  approaches  to  the  project. 

The  series  strategy  is  represented  by  path  variables  15  through  17 
in  Figure  4-1.  For  the  series  strategy,  one  of  the  two  approaches  is 
selected  for  research  funding  (Abernathy  and  Rosenbloom  suggest  the 
preferred  approach,  which  is  defined  as  the  "apriori  best  evident” 
approach,  be  initiated  first^).  In  this  case,  let  the  preferred 
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approach  be  10A,  represented  by  path  variable  15  in  the  figure,  as, 
even  though  the  probability  of  success  is  lower  than  10B,  a  substanially 
smaller  budget  commitment  is  required  in  the  first  period.  If  this 
approach  is  successful,  it  is  carried  on  to  completion.  If  not, 
research  work  is  initiated  on  approach  10B,  indicated  as  path  variable 
16  in  Figure  4-1.  Thus,  the  series  strategy  could  result  in  a  three 
year  time  commitment  if  the  first  approach  fails.  Note  that  path  vari¬ 
able  17  is  included  to  represent  the  case  where  both  approaches  have 
failed  and  the  research  work  is  unsuccessfully  terminated. 

The  alternative  strategy  is  to  initiate  research  work  on  both 
approaches  simultaneously  and  continue  funding  that  version  which 
indicates  technical  success  after  the  first  year.  Thus,  as  shown  by 
path  variable  18  in  Figure  4-1,  the  first  year  resource  commitment  is 
the  sums  of  the  budget  and  staff  required  for  both  approaches.  After 
the  first  year  in  development,  four  outcomes  are  possible  and  shown  in 
the  project  tree: 

1)  Future  event  K:  Approach  10A  is  successful,  10B  is  not. 

The  probability  of  this  outcome  is:^ 

PlOA  (1-»10B>  '  •“•2)  ’  '°8  <4-2) 

2)  Future  event  L:  Approach  10B  is  successful,  10A  is  not. 


The  probability  of  this  outcome  is: 


»10B  (1-'>10A)  '  •“  ('6>  '  •“ 


(4.3) 


3)  Future  event  M:  Both  approaches  are  successful.  The 


probability  of  this  outcome  is: 

p  p  =  ,4  (.8)  =  .32 
10A  10B 


(4.4) 
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4)  Future  event  N:  Both  versions  fail.  The  probability  of 
this  outcome  is: 

H-P10A)  (1-P10B>  '  -f’  ('2)  '  -12  M'5) 

If  future  event  M  occurs,  a  decision  node  is  shown  at  the  beginning  of 
the  second  year,  since  both  approaches  10A  and  10B  indicate  technical 
successes  after  the  first  year  and  a  choice  must  be  made  as  to  which 
approach  to  continue.  With  the  same  reward  anticipated,  it  would  seem 
that  the  correct  choice  would  be  approach  10A  (path  variable  20)  since 
the  resource  commitment  is  less  than  that  required  for  10B.  However, 
one  of  the  multiple  goals  (to  be  described  in  the  following  section) 
is  to  minimize  the  underutilization  of  the  14  scientists  in  the  second 
year  and  it  may  be  advantageous  to  select  the  approach  requiring  one 
more  scientist  under  certain  situations,  such  as  the  need  to  pursue 
some  sort  of  work  force  smoothing  policy. 

Zero-One  Goal  Programming  Model  Formulation 

Tvto  experiments  will  be  presented  to  illustrate  the  potential 
capabilities  of  the  zero-one  goal  program  as  a  decision  aiding  mecha¬ 
nism.  The  first  concerns  analysis  of  portfolios  and  associated  goal 
attainment  levels  for  several  of  the  future  outcome  states  of  interest 
to  the  model  users.  The  second  concerns  analysis  of  results  when  goal 
priority  levels  are  changed.  In  each  experiment,  the  technique  of  simu¬ 
lating  dominant  future  outcome  states  is  used  to  select  the  path  vari¬ 
ables  (associated  with  the  chance  nodes)  for  input  into  the  zero-one 
goal  programming  formulation.  This  method  is  chosen  rather  than  using 
a  stochastic  goal  programming  formulation,  because,  with  96  future 
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states  possible  over  two  time  periods,  each  having  two  resources  to 
account  for,  the  number  of  model  constraints  and  variables  becomes 
unwieldy  in  a  stochastic  formulation.  In  addition,  aside  from  the 
size  problem,  simulation,  or  forced  future  state,  selections  enable 
the  decision  makers  to  analyze  strategies  associated  with  dominant  or 
otherwise  selected  future  outcomes  as  has  been  previously  discussed. 
Both  experiments  will  also  involve  simulation  studies  to  determine  how 
sensitive  the  various  portfolio  strategies  selected  by  the  zero-one 
goal  programming  algorithm  are  to  a  range  of  values  for  selected 
system  parameters, 

A  complete  goal  programming  model,  as  outlined  in  other  applica- 

8 

tions,  is  developed  in  the  following  steps: 

1)  Identify  decision  and  deviational  variables. 

2)  Establish  model  objectives  and  their  priorities. 

3)  Formulate  goal  and  system  constraints. 

A)  Analyze  model  output. 

Since  the  R&D  selection  problem  is  to  determine  which  projects  to 
include  in  the  research  portfolio,  the  decision  variables  are: 

1  if  path  variable  j  is  selected 
Xj=<  (A. 6) 

0  otherwise. 

The  path  variable  notation  has  been  adopted  for  ease  of  representation. 
Reference  is  made  to  Figure  A-l  for  interpretation  of  the  x^'s.  For 
example,  represents  version  3  of  project  10,  given  that  future 
event  M  is  under  consideration  or  has  occurred  (from  the  original 


description  of  project  10,  this  also  represents  approach  10A).  The 


deviational  variables  are  n_^,  representing  negative  deviation  from 
goal  i,  and  p^,  representing  positive  deviation. 

For  the  second  and  third  steps,  the  following  goals  and  related 
constraint  equations,  shown  in  Table  4-1,  are  developed.  In  this 
illustration,  the  constraint  equations  are  for  future  outcome  state 
BCFHIM  and  are  typical  of  the  constraint  equations  for  other  future 
states  used  in  Experiments  1  and  2. 

Priority  1:  Ensure  that  only  one  of  the  mutually  exclusive  ver¬ 
sions  of  a  project  is  selected  if  the  project  is  included  in  the  final 
portfolio.  This  goal,  although  considered  a  system  constraint  to  ensure 
solution  feasibility,  represents  the  most  important  objective  since 
unplanned  duplication  of  effort  should  be  avoided  in  any  final  port¬ 
folio.  The  goal  is  achieved  by  minimizing  the  negative  deviational 
variables  in  rows  1  -  3  of  Table  4-1. 

Priority  2:  Limit  the  research  portfolio  to  no  more  than  four 
projects  expected  to  take  more  than  two  years  in  the  research  and 
development  phase.  This  goal  reflects  the  decision  maker’s  interest 
in  achieving  a  balance  betvjeen  long  and  short  run  projects.  It  is 

achieved  by  minimizing  p  in  the  row  4  equation. 

4 

Priority  3:  Limit  the  portfolio  to  no  more  than  two  projects  with 
an  expected  probability  of  success  less  than  .30.  This  goal  quantifies 
the  desire  to  limit  the  number  of  risky,  high  return  projects  in  the 
portfolio.  It  is  satisfied  by  minimizing  p^  in  the  row  3  constraint 
equation. 

Priority  4:  Maximize  the  expected  benefit  of  the  portfolio.  The 
related  constraint  equation  for  this  goal  is  shown  in  row  6  and  the  goal 
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is  achieved  by  minimizing  n  from  a  large,  unattainable  RHS  value  of 

6 

$  100,000,000.  Note  that  the  technological  coefficients  are  the  respec¬ 
tive  probabilities  of  success  times  the  benefit  values  from  Figure  4-1. 

Priority  5:  Minimize  yearly  budget  overruns.  This  goal  is  repre¬ 
sented  by  three  constraint  equations  shown  as  rows  7-9,  and  is  satis¬ 
fied  by  minimizing  p  ,  p  ,  and  p  from  the  respective  RHS  total  budget 

7  8  9 

levels. 

Priority  6;  Minimize  underutilization  of  the  scientific  staff  in 
each  time  period.  This  goal  is  achieved  by  minimizing  the  negative 
deviational  variables  in  rows  10  through  12. 

Priority  7 :  Maximize  the  total  expected  percent  increase  in  market 
share.  This  goal  is  achieved  by  minimizing  n  in  row  13.  Note  that 
the  technological  coefficients  are  the  probabilities  of  commercial 
success  times  the  respective  market  share  increase  values  for  each 
variable  as  shown  in  Figure  4-1. 

The  objective  function  for  this  case  study  example  includes  mini¬ 
mizing  the  deviations  from  the  set  goals  with  established  preemptive 
priority  factors  such  that  (p^>P2> _ >Pk^ ; 


Minimize  a  * 


pi(pi  +  p2  +  p3);  p3(p5); 


W;  V3p7  +  2p8  +  p9); 


P6^3n10  +  2nll  +  n12^;  P7^n13^ 


(4.7) 


Note  that  weights  have  been  assigned  to  the  positive  and  negative 
deviational  variables  at  priority  levels  5  and  6  to  ensure  that  earlier 
year  resource  utilization  goals  are  satisfied  before  later  year  goals. 
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The  rationale  for  this  within  priority  weighting  is  that  less  flexi¬ 
bility  for  budget  or  personnel  adjustments  exists  in  the  earlier  years. 
In  actual  practice,  R&D  decision  makers  would  be  responsible  for 
developing  relative  within  priority  weighting  schemes  as  appropriate 
for  the  situation.  The  example  is  used  here  to  illustrate  that  cardi¬ 
nal  rankings  are  possible  for  commensurate  subgoals  in  goal  programming 
formulations . ^ 

Experiment  1:  Dominant  Future  State  Analysis 

For  this  experiment,  the  decision  makers  are  interested  in 
examining  the  different  goal  attainment  levels  resulting  from  the 
portfolios  associated  with  different  future  event  scenarios  repre¬ 
sented  by  the  future  states  previously  described  in  the  Figure  4-1 
project  trees.  Six  of  the  96  possible  future  states  have  a  combined 
theoretical  probability  of  occurrence  of  37  percent  and  were  chosen  as 
the  future  states  most  likely  to  occur,  and  therefore,  are  of  interest 
to  the  decision  makers.  Six  runs  of  the  zero-one  goal  programming 
computer  program  were  made,  one  for  each  future  state  parameter  inputs. 
The  results  of  the  six  runs,  identified  as  benchmark  models  1  through 
6,  are  shown  in  Table  4-2. 

The  next  step  involves  a  simulation  study  to  determine  how 
sensitive  the  various  portfolio  strategies  are  to  a  range  of  values 
for  the  system  parameters.  For  this  experiment,  only  the  benefit  values 
vary  although  other  variables,  such  as  the  probabilities  of  commercial 
success  and  resource  availabilities  can  be  included  as  non-de terminist ic 
parameters.  The  hypothesis  to  be  tested  involves  determining  whether 
or  not  the  different  portfolios  generated  for  the  dominant  future  states 


Table  4-2 

Experiment  1:  Portfolios  and  Goal  Attainment  Levels 
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of  interest  result  in  significantly  different  goal  attainment  levels 
for  the  fourth  priority  objective  (which  represents  the  maximization  of 
benefit  values  that  are  now  input  as  non-deterministic  parameters;  and 
which  also  represents  the  highest  priority  goal  which  might  be  ade¬ 
quately  compared  since  the  first  three  goals  were  attained  in  each 
model  as  shown  in  Table  4-2) .  For  this  step,  minimum  and  maximum 
benefit  values  are  estimated  for  each  project  version  as  shown  in  Table 
4-3.  When  nothing  except  the  range  values  are  known,  Chan^  suggests 
the  uniform  distribution  be  used  to  generate  parameter  values  as  this 
maximizes  the  uncertainty  of  the  simulated  environment.  Accordingly, 
a  computer  code  was  written  in  Fortran  IV  to  generate  uniformly  distri¬ 
buted  benefit  values  and  calculate  the  fourth  priority  goal  attainment 
for  each  of  the  six  models  and  associated  portfolios.  This  process  was 
repeated  for  30  observations  of  each  model  using  six  different  seeds 
for  the  IMSL  random  number  generator  used.^1  Table  C-l,  Appendix  C, 
contains  the  observations,  means  and  variances  of  the  benefits  achieved 
for  each  model.  The  mean  benefit  values  are  also  provided  in  Table  4-4 

shown  on  page  81.  It  is  noted  that  in  addition  to  range  distributions, 

12  13  14 

Asher,  Elnicki,  and  Cochran,  et.  al. ,  found  that  R&D  decision 

makers  often  estimate  empirical  distributions  for  the  benefit  parameters 

by  providing  the  pessimistic,  most  likely  and  optimistic  estimates  and 

their  associated  probabilities.  lhese  empirical  or  any  other  type  of 

statistical  distributions  could  be  represented  in  the  project  trees  and 

simulation  used  to  generate  benefit  values  based  upon  the  distributions, 

as  was  done  in  Table  4-4. 

A  one-way  factor  analysis  of  variance  test  was  used  to  determine 
if  there  is  a  significant  difference  in  the  benefits  achieved  for  each 
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Table  4-3 
Expe  r iment  1 
Benefit  Values 


Path 

Benefit  Range 

(106  $) 

Variables 

Minimum 

Maximum 

1 

3 

7 

2 

3 

5 

3 

4 

10 

4 

2 

6 

5 

5 

11 

6 

5 

15 

7 

6 

12 

8 

16 

32 

9 

7 

11 

10 

3 

12 

11 

12 

18 

12 

0 

0 

13 

8 

12 

14 

7 

13 

15 

7 

7 

16 

7 

7 

17 

0 

0 

18 

7 

7 

19 

7 

7 

20 

7 

7 

21 

7 

7 

22 

0 

0 

Table  4-4 

Mean  Benefit  Values 


(106  $) 


Model  1 
49.83 


Model  2 
50.43 


Model  3 
44.88 


Model  4 
49.55 


Model  5 
44.50 


Model  6 
51.91 


model.  The  ANOVA  table  appears  as  Table  C-2 ,  Appendix  C,  and  the 
conclusion  is  that  there  exists  a  significant  difference  between  the 
mean  benefit  values  of  the  six  models.  The  next  test  is  to  determine 
which  means  are  different  through  multiple  comparison  analysis.  The 
multiple  comparison  test,  reported  in  Table  03,  Appendix  C,  resulted 
in  no  difference  between  the  means  of  models  1,  2,  4  and  6  and  no 
difference  between  the  mean  benefits  of  models  3  and  5.  However,  the 
mean  benefits  of  the  first  group  are  all  significantly  higher  than 
those  of  the  second.  At  this  point,  it  is  concluded  that  different 
portfolio  strategies  are  suggested  according  to  which  of  the  dominant 
future  states  occur,  but  four  of  the  strategies  would  result  in  attain¬ 
ment  levels  which  are  statistically  indistinguishable  for  the  four 
highest  priority  goals.  Further  analysis  of  these  four  portfolios  and 
associated  models  reveals  two  major  decisions  occurring  at  the  com¬ 
pletion  of  the  first  year;  namely,  the  decision  to  select  path  variable 
15  or  16  (assuming  that  path  variable  17  is  the  result  of  unanticipated 


events  occurring  during  the  second  year),  and  the  decision  to  select 
path  variable  19  or  20.  Referring  to  Table  4-2,  either  of  these 
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strategies  would  result  in  the  same  attainment  levels  for  the  sixth 
and  seventh  priority  goals  but  the  portfolio  associated  with  models  2 
and  6  outperforms  on  the  budget  goal. 

These  results  are  then  fed  back  to  the  decision  makers  respon¬ 
sible  for  proposal  data  generation  and  to  those  responsible  for  final 
project  selection.  The  benefit  of  this  approach  is  that  decision 
makers  can  have  the  model  generate  portfolios  and  goal  attainment 
levels  for  selected  future  technological  events.  Trade-offs  can  be 
analyzed  and  the  interactive  flow  between  decision  makers  and  the 
model  continues  until  consensus  is  reached  with  respect  to  future  out¬ 
come  analysis  and  portfolio  selection.  For  the  problem  at  hand,  the 
portfolio  associated  with  models  2  and  6  results  in  the  highest  goal 
attainment  levels.  The  other  portfolio  leads  to  expected  budget  over¬ 
runs  and  since  authoritative  sources  such  as  Meadows, ^  Peck  and 
Scherer, ^  and  Mansfield,  et.  al.^  report  cost  overruns  as  common 
place  in  the  R&D  environment,  planned  overruns  are  normally  avoided. 

Experiment  2;  Priority  Structure  Analysis 

The  dilemma  at  the  completion  of  Experiment  1  is  that  the  port¬ 
folio  associated  with  model  1  (which  leads  to  a  planned  budge L  overrun) 
is  the  portfolio  generated  by  the  goal  programming  algorithm  using  as 
input  the  future  state  most  likely  to  occur,  namely  BCFHIL.  It  may  be 
of  interest,  therefore,  to  determine  what  portfolio  strategy  is  sug¬ 
gested  by  the  algorithm  for  this  future  state  if  the  budget  goal  is 
made  a  higher  priority  goal. 

This  type  of  analysis  is  also  suggested  when  the  decision  makers 
cannot,  after  several  pairwise  comparison  experiments,  achieve  a 
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comfortable  degree  of  consensus  on  the  priorities  of  the  six  estab¬ 
lished  multiple  objectives.  Thus,  this  experiment  could  also  represent 
the  case  where,  for  example,  the  scientists  favored  the  priority 
structure  used  in  the  conduct  of  Experiment  1,  whereas  the  laboratory 
director  and  division  chiefs  placed  more  importance  on  minimizing 

budget  overruns,  a  situation  which  can  be  expected  from  the  value 

18 

differences  existing  between  these  two  groups. 

The  new  priority  structure  for  this  experiment  therefore  places 
more  importance  on  the  budget  goal  and  the  complete  set  of  revised 
"administrators1 "  priorities  is  as  follows: 

Priority  1:  Ensure  that  only  one  of  the  mutually  exclusive 
versions  of  a  project  is  selected  if  the  project  is  included  in  the 
final  portfolio. 

Priority  2:  Limit  the  research  portfolio  to  no  more  than  four 
projects  expected  to  take  more  than  two  years  in  the  research  and 
development  phase. 

Priority  3:  Limit  the  portfolio  to  no  more  than  two  projects  with 
an  expected  probability  of  success  less  than  .50. 

Priority  4:  Minimize  yearly  budget  overruns. 

Priority  5:  Maximize  the  expected  benefit  of  the  portfolio. 

Priority  6:  Minimize  underutilization  of  the  scientific  staff. 

Priority  7:  Maximize  the  total  expected  percent  increase  in 
market  share. 

The  hypothesis  to  be  tested  concerns  whether  or  not  different 
portfolios  with  different  goal  attainment  levels  are  suggested  by  the 
different  priority  structures.  For  this  experiment,  another  run  of 
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the  goal  programming  algorithm  was  made  using  the  same  future  state 
and  other  input  data  as  was  used  for  model  1  in  Experiment  1,  except 
for  changing  the  priority  structure  to  the  one  favored  by  the  labora¬ 
tory  management  group.  The  results  of  this  new  run  are  shown  under 
Model  1A  of  Table  4-5  along  with  the  original  model  1  results  from 
Experiment  1  (Table  4-2).  As  can  be  seen  here,  the  revised  priority 
structure  results  in  a  different  portfolio  and,  consequently,  different 
goal  attainment  levels. 

The  next  step  in  this  experiment  is  a  simulation  study  used  to 
determine  how  sensitive  the  portfolio  strategies  are  to  a  range  of 
values  for  the  system  parameters.  The  benefit  values  are  allowed  to 
vary  over  the  ranges  previously  shown  in  Table  4-3.  The  same  computer 
code  used  in  Experiment  1  was  used  to  generate  uniformly  distributed 
benefit  values  and  calculate  the  fourth  priority  goal  attainment  level 
for  model  1  and  the  fifth  priority  goal  attainment  level  for  model  1A. 
This  process  was  repeated  for  30  observations  of  each  model  using  the 
same  random  number  generator  seed  for  the  generator  used.  In  this 
manner,  observations  were  matched  in  the  sense  that  internally  generated 
random  numbers  and  seeds  were  the  same,  the  only  difference  being  the 
different  portfolios  for  the  two  priority  structures.  The  observations 
on  the  benefit  functions  and  statistical  analysis  are  reported  in 
Appendix  D.  The  mean  expected  benefit  of  model  1  is  $  50,716,000, 
which  is  significantly  greater  than  that  of  model  1A  which  is  $  48,895, 
000.  This  information,  together  with  a  comparison  of  the  other  goal 
attainment  levels  shown  in  Table  4-5,  would  now  be  used  by  the  decision 
makers  for  further  analysis. 
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Table  4-5 
Experiment  2 

Portfolios  and  Goal  Attainment  Levels 


Model 

Future  State 
Path  Variables 


1 

BCFHIL 

1-3-5-8 

10-11-14-19 


1A 

BCFHIL 

3-4-5-8 

10-11-14-19 


Benefit  Attained  (10  $) :  50.0 

Budget  Overruns  (10"*  $)  : 

t=l  2 

t=2  1 

t=3  0 


48.2 


0 

0 

0 


Idle  Staff: 


t=l 

t*2 

t=3 


0 

0 

4 


0 

0 

5 


%  Increase  in  Market  Shr: 


3.375 


3.925 


aThe  first  three  goals  were  reached  in  each  model. 


Using  the  path  variable  notation  from  Figure  3-4. 


In  actual  applications  where  consensus  concerning  the  goal  priority 
levels  could  be  blocked,  and  the  integer  goal  program  is  used  to  gener¬ 
ate  alternate  portfolios,  several  outcomes  are  possible.  First,  a 
common  portfolio  could  emerge  for  the  different  priority  sets  in  which 
case  the  problem  of  blocked  consensus  becomes  moot  since  the  same  port¬ 
folio  is  suggested  no  matter  which  priority  structure  is  used.  The 
second  outcome  could  be  that  dominant  portfolios  emerge  for  each 
priority  set  but  the  goal  attainment  levels  are  not  significantly 
different.  Again,  the  problem  of  blocked  consensus  is  overcome  by 
model  results.  The  third  outcome  could  be  the  case  where  dominant 
portfolios  emerge  for  each  priority  structure  and  the  respective  goal 
attainment  levels  are  significantly  different.  This  outcome  occurred 
in  this  experiment.  The  significantly  higher  benefit  of  model  1 
requires,  as  expected,  an  increase  in  the  funding  level  of  $  300,000  in 
the  first  two  years.  Model  1A  reflects  one  more  scientist  idle  in 
year  3  but,  according  to  the  weights,  this  is  not  a  critical  time 
period.  Finally,  the  portfolio  associated  with  model  1A  reflects  a 
higher  expected  increase  in  the  expected  market  share.  The  quantitative 
analysis  suggests  the  model  1A  portfolio  as  the  favored  strategy  since 
the  incremental  expected  return  of  model  1  is  substantially  less  than 
the  return  of  model  1A.  The  final  decision,  however,  rests  with  the 
decision  makers  and  all  results  are  provided  to  them  for  further 
analysis.  The  benefit  of  using,  the  model  results  in  interaction  with 
the  decision  makers  is  to  afford  them  suggested  strategies  and  goal 
attainment  levels  for  the  alternatives  presented.  In  the  case  study, 
if  the  management  and  staff  groups  do  not  achieve  consensus  in  select¬ 
ing  one  of  the  two  portfolios,  the  last  resort  is  to  examine  the 


dominant  projects  that  appear  in  the  two  portfolios  of  Table  4-5, 
Projects  3,  5  (version  1),  6  (version  2),  7,  8,  9  and  10  (version  2) 
should  be  selected  no  matter  which  priority  structure  is  agreed  upon. 
The  second  priority  structure  suggests  project  4  should  be  included 
whereas  project  1  is  suggested  by  the  other  structure.  To  assist  in 
the  final  selection,  qualitative  techniques  such  as  paired  comparison 
experiments,^  may  be  utilized. 
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to  the  zero-one  goal  programming  algorithm  and  the  portfolio  suggested 
by  running  the  program  was  recorded.  This  procedure  was  repeated  for 
30  observations  and  the  project  frequencies  in  the  output  results  are 
shown  in  Table  4-6.  Project  1,  5  (version  1),  6  (version  2),  7,  8  and 
9  are  dominant  and  should  be  included  in  the  final  portfolio.  The 
remaining  problem  is:  which  of  the  projects  not  yet  selected  should 
be  included  in  the  final  portfolio? 

A  disadvantage  of  this  approach  is  that  the  project  selection 
problem  may  not  be  solved  at  the  completion  of  the  programming  model 
runs  and  an  alternative  approach  is  required  to  complete  the  portfolio. 
This  was  the  case  in  the  above  illustration. 


Table  4-6 

Dominant  Project  Frequencies 
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Projects  Versions  Path 

Variables 


1  1 

2  2 

3  3 

4  4 

5  1  5 

2  6 

6  1  7 

2  8 

7  9 

10 

8  11 

12 

9  13 

14 

10  1  15 

1  16 

1  17 

2  18 

2  19 

3  20 

4  21 

2  22 


Number  of  Times  Path  Variable 
Appeared  in  Selected  Portfolio 

28 

3 

22 

17 

21 

9 

1 

29 

10 

20 

16 

4 

4 

26 

14 

1 

0 

1 

9 

5 
0 
0 
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CHAPTER  V 

MODEL  APPLICATIONS 

While  the  purpose  of  the  previous  chapter  was  to  examine  the 
conceptual  utility  of  the  proposed  model,  the  emphasis  in  this  chapter 
is  on  the  practical  utility  of  the  model.  This  will  be  performed  by 
examining  the  efficiency  and  validity  of  the  proposed  model  through 
comparison  of  its  results  with  those  of  other  state-of-the-art  mathe¬ 
matical  programming  models.  Three  previously  published  R&D  project 
selection  problems,  two  of  which  represent  actual  studies,  are  used 
for  the  data  input.  In  addition,  to  further  assess  the  feasibility 
of  the  model  and  gain  a  broader  insight  into  actual  decision  making 
processes,  interview  sessions  were  conducted  with  R&D  administrators 
of  five  diverse  enterprises,  in  order  to  discuss  the  proposed  multiple 
objective  project  selection  and  resource  allocation  model  in  the 
context  of  the  R&D  decision  making  process  unique  to  each  of  the  five 
organizations.  Specifically,  the  nature  of  the  project  selection 
decision  was  examined  with  regards  to  the  realism,  applicability, 
feasibility  and  limitations  of  the  proposed  formal  decision  aiding 
tool.  Finally,  as  an  outcome  of  the  interview  with  the  director  of 
the  research  laboratory  of  a  large  textile  firm,  an  actual  application 
of  the  model  was  developed  and  is  presented. 
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Efficiency  and  Validity  Comparisons 

Application  1;  A  Static  Single  Objective  Case 

Most  of  the  quantitative  models  proposed  for  the  R&D  project 
selection  problem  involve  linear  or  2ero-one  programming  formula¬ 
tions  for  the  single  objective,  single  period,  case.  Although  the 
proposed  model,  hereafter  referred  to  as  the  thesis  model,  waa  designed 
to  incorporate  multiple  objectives  and  multiple  periods  with  uncer¬ 
tainty,  it  must  be  capable  of  providing  good  solutions  to  the  specific 
class  of  single  objective  problems  in  order  to  enhance  its  general 
applicability  # 

To  test  the  validity"^  and  efficiency  of  the  thesis  model,  a  ''real 

2 

world”  problem  presented  by  Souder"  was  selected  and  solved  using  both 

the  thesis  model  and  the  efficient  zero-one  algorithm  developed  by 
3 

Mandakovic.  The  model  proposed  by  Mandakovic  was  chosen  as  represen¬ 
tative  of  the  state-of-the-art  algorithms  for  single  objective,  integer 

4 

programming  formulations  with  zero-one  variables.  Mandakovic  devel¬ 
oped  two  algorithms  in  his  research,  both  of  which  employ  the  Senju- 
Toyoda^  and  Tovoda^  primal  effective  gradient  method  for  solving  formu¬ 
lations  of  the  type  shown  by  equations  (2.1)  -  (2.6)  in  Chapter  II. 
Mandakovic's  initial  algorithm  involves  selection  of  that  project  or 
project  version  having  the  maximum  index  of  profitability  for  a  given 
level  of  resource(s),  deducting  the  resources  consumed  by  the  selected 
project  from  the  total  available,  and  repeating  the  selection  procedure 
until  the  critical  resource  is  exhausted.  The  index  of  profitability 
is  defined  as  the  rate  of  return  per  unit  of  resource  used.  The  second 
algorithm  developed  by  Mandakovic  was  specifically  designed  for  hierar¬ 
chical  R&D  organizations  and  decomposes  the  selection  problem  into  two 
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parts;  one  for  the  superordinate  level  and  the  other  for  the  subordi¬ 
nate  level.  An  interactive  mode  is  used  for  communication  of  budget 
levels  (from  superordinate  to  subordinate)  and  selected  project  ver¬ 
sions  (from  the  subordinate  to  superordinate).  Both  algorithms,  being 
based  on  heuristic  techniques,  were  designed  to  rapidly  produce  good 
results  for  medium  to  very  large  scale  problems. ^ 

g 

The  input  data  from  Souder  involves  a  30  project  problem  solved 
in  this  case  for  the  start  of  the  budget  period.  The  maximum  funding 
levels  for  each  project  and  the  maximum  expected  project  values  are 
shown  in  Table  5-1.  Each  project  can  be  selected  in  one  of  three 
versions;  where  the  first  version  represents  a  project  consuming  25% 
of  the  maximum  funding  level;  the  second,  a  50%  funding  level;  and 
the  third,  the  maximum  funding  level.  Associated  with  each  of  the  90 
resulting  project  versions  is  a  probability  of  success  shown  in  Table 
5-2  which,  when  multiplied  by  the  project  value,  gives  the  expected 
gross  return  for  the  particular  version.  The  objective  is  to  maximize 
the  total  expected  return  of  the  portfolio  chosen  subject  to  the  budget 
constraint,  mutually  exclusive  project  version  constraints,  and  the 
zero-one  variable  restrictions. 

Souder’ s  problem  was  decomposed  into  nine  subproblems  by  varying 
the  number  of  variables  and  the  total  budget  available  as  shown  in 
Part  1  of  Table  5-3.  Subproblem  1  includes  the  first  10  projects  and 
their  respective  versions,  funding  levels  and  expected  returns,  sub¬ 
problem  2  includes  the  first  15  projects  and  associated  parameters,  and 
so  on  until  all  30  projects  are  included  in  subproblems  7,  8  and  9. 

The  budget  level  was  set  at  approximately  45%  of  the  sum  of  the  project 
maximum  funding  levels  for  each  subnroblem  except  3,  5,8  and  9  which 


Table  5-1 

Data  for  Application  1 
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Project  Maximum  Funding  Project 

Number  Level  ($  000)  Value  ($  000) 


1 

300 

11,600 

2 

600 

16,200 

3 

700 

12,000 

4 

250 

10,500 

5 

200 

22,900 

6 

450 

13,100 

7 

300 

15,600 

8 

500 

20,500 

9 

200 

11,500 

10 

200 

13,500 

11 

300 

22,100 

12 

300 

5,500 

13 

150 

15,500 

14 

500 

20,000 

15 

200 

7,500 

16 

350 

12,000 

17 

150 

20,000 

18 

500 

8,500 

19 

100 

12,000 

20 

300 

8,500 

21 

200 

21,500 

22 

100 

11,800 

23 

400 

18,500 

24 

350 

7,000 

25 

600 

14,000 

26 

600 

20,500 

27 

1 00 

6,000 

28 

250 

12,500 

29 

600 

18,100 

30 

400 

5,500 

I 
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Project 

Number 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 


Table  5-2 

Probabilities  of  Success  for 
the  Various  Funding  Levels 
Application  1 


Version  1  Version  2 

(pp  <p2> 


Version  3 

(p3) 


.20 

.30 

.50 

.20 

.40 

.70 

.30 

.30 

.65 

.10 

.25 

.80 

.10 

.50 

.75 

.30 

.70 

.65 

.30 

.50 

.90 

.20 

.55 

.90 

.25 

.30 

.50 

.05 

*  35 

.80 

.10 

.50 

.70 

.10 

.20 

.60 

.10 

.20 

.60 

.00 

.10 

.50 

.10 

.20 

.70 

.10 

.30 

.70 

.10 

.  30 

.70 

.10 

.20 

.60 

.05 

.10 

.50 

.10 

.30 

.70 

.10 

.70 

.80 

.30 

.80 

.90 

.10 

.70 

.80 

.30 

.80 

.90 

.30 

.90 

1.00 

.40 

.30 

.90 

.40 

.70 

.80 

.20 

.80 

.90 

.30 

.90 

.90 

.30 

.80 

.90 

Table  5-3 

Model  Comparisons:  Application 
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Results  Reported  to  Nearest  $  Million 


examine  tighter  (subproblem  3)  or  relaxed  (subproblems  5,  8  and  9) 

levels.  Although  Mandakovic  solved  subproblem s  7,  8  and  9  and  reported 

9 

the  results  in  his  dissertation,  a  computer  code  for  his  initial  algo¬ 
rithm  was  written  for  these  single  resource  constrained,  single  time 

period  problems  in  order  to  obtain  results  for  subproblems  1  through  6 

10 

and  CPU  times  for  all  of  the  problems  solved.  For  the  thesis  model, 
the  single  objective  subproblems  were  converted  into  multiple  objective 
formulations  having  three  goals.  The  first  goal  was  to  ensure  that 
either  one  or  none  of  the  three  versions  of  each  project  is  selected. 

The  second  goal  was  to  stay  within  the  budget  constraint  and  the  third 
goal  was  to  maximize  the  return  of  the  portfolio  selected.  The  vari¬ 
able  selection  and  exchange  heuristics  (option  number  2,  as  described 
in  Chapter  III  and  in  Appendix  A)  was  used  to  solve  all  nine  sub¬ 
problem  formulations. 

Parts  2  and  3  of  Table  3-3  provide  the  comparisons  between  the 
two  models.  In  every  case,  the  thesis  model  outperformed  the  Manda¬ 
kovic  algorithm  with  respect  to  expected  net  return  (sum  of  the 
expected  values  less  funds  expended).  The  average  improvement  in  net 
returns  was  8%  with  a  range  between  .  3?'  and  20%.  The  validity  of  the 
thesis  model  with  respect  to  this  test  appears  to  be  established.  As 
expected,  the  Mandakovic  algorithm  was  very  efficient,  solving  the 
various  subproblems  in  .2  to  .5  seconds.  In  contrast,  the  thesis  model 
required  from  2.8  to  313.6  seconds.  The  tradeoff  between  the  heuristic 
models  is  thus  one  of  solution  performance  versus  computer  time  and 
cost  for  medium  size  problem  applications.  As  an  extreme  example, 
subproblem  6  illustrates  a  strategy  with  an  expected  return  of 
$  28,165,000  more  for  the  thesis  model  at  a  cost  increase  of  20.5  seconds 


While  there  is  no  theoretical  limit  to  the  size  problem  which  the 
thesis  model  can  address,  some  practical  constraints  are  imposed  by 
operating  time  requirements  and  serve  to  limit  the  size  problem  which 
the  model  can  realistically  consider.  However,  the  range  of  capability 
of  the  thesis  model  does  include  most  of  the  realistic  size  problems 
encountered  in  the  R&D  environment.  For  multiple  objective  problems 
with  more  than  100  variables,  it  was  suggested  in  Chapter  III  that 
the  variable  selection  heuristic  can  be  used  without  the  exchange 
routine  when  CPU  time  is  an  important  consideration.  A  180  variable 
problem  similar  in  structure  to  the  90  variable  problems  reported  in 
this  application  was  created  to  test  the  efficiency  and  validity  of 
the  thesis  model  using  only  the  variable  selection  procedure.  The 
problem  was  also  solved  by  the  Mandakovic  algorithm  for  comparison 
purposes.  The  thesis  model  required  21  seconds  of  CPU  time  compared 
to  .9  seconus  for  the  Mandakovic  algorithm;  and  the  solution  obtained 
by  the  thesis  model  was  7.7%  less  than  the  solution  obtained  by  the 
Mandakovic  algorithm.  The  variable  selection  procedure  does  provide 
an  alternative  for  larger  size  problems  and,  in  addition,  represents 
a  more  direct  and  realistic  approach  to  addressing  multiple  objectives 
than  do  the  single  objective  models. 

To  complete  the  validity  evaluation.  Part  4  of  Table  5-3  provides 

Souder's  original  results  for  subproblems  7,  8  and  9  using  zero-one, 

linear  and  nonlinear  programming  algorithms . These  results  are 

reported  here  as  they  were  used  by  Mandakovic  to  test  the  validity  of 

12 

his  zero-one  algorithm.  The  thesis  model  outperformed  the  zero-one 
and  linear  models  in  every  case  and  equalled  the  nonlinear  solution  for 


100 

subproblem  7.  This  comparison  must  be  cautiously  considered,  however, 
since  Souder fs  zero-one  model  only  included  version  3  of  each  project 
and  the  linear  and  nonlinear  optimization  was  made  on  continuous  vari¬ 
ables  with  the  versions  being  approximated.  Thus,  according  to 
Mandakovic,  the  linear  and  nonlinear  solutions  are  in  the  superop timal 

infeasible  area  when  considering  that  the  zero-one  restrictions  repre- 

13 

sent  the  realistic  case. 

Application  2:  A  Multiperiod,  Multiple  Objective  (Dynamic  Programming) 
Case 

The  second  application  involved  a  validity  test  of  the  thesis 

model  using  a  dynamic,  noncommensurate  multiple  objective  R&D  project 

selection  problem.  The  state-of-the-art  model  chosen  for  comparison 

14 

purposes  is  the  dynamic  programming  technique  of  Rosen  and  Souder. 

Up  until  the  introduction  of  decision  tree  modeling,  the  dynamic 
programming  approach  was  the  only  method  capable  of  addressing  the 
multistage  nature  of  project  evolution  along  with  the  considerat ion  of 
uncertainty,  Rosen  and  Souder  further  attempted  to  enhance  the  realism 
of  their  formulation  by  recognizing  the  existence  of  multiple  objec¬ 
tives,  albeit  in  an  indirect  manner.  Their  method  includes  the 
recognition  of  a  set  of  multiple  criteria  for  a  particular  case  study, 
one  of  which  is  chosen  for  optimization  in  their  single  objective 
dynamic  programming  model.  Then,  using  the  selected  portfolio 
suggested  by  the  model,  the  goal  attainment  levels  for  the  other 
objectives  are  calculated.  If  any  of  the  goals  are  not  attained, 
the  model  is  rerun  with  one  or  more  relaxed  objectives;  with  a 
reduced  set  of  project  versions;  or  with  a  modification  of  the 
constraints. 
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The  problem  used  to  test  the  validity  of  the  thesis  model  with 
respect  to  the  dynamic  programming  technique  is  a  smaller  version  of 
the  problem  reported  by  Rosen  and  Souder  in  the  publication  noted 
above. ^  This  problem  is  an  illustration  of  the  type  of  project 
selection  problems  encountered  at  the  Monsanto  Chemical  Company  at 
the  time  of  publication.  Table  5-4  shows  the  input  data  for  six  pro¬ 
jects  being  considered  for  funding  in  this  case  study.  Note  that  only 
the  first  year  budget  constraint  is  specified  for  the  three  period 
planning  horizon.  This  is  necessary  due  to  the  dimensionality  restric¬ 
tions  of  dynamic  programming.  However,  some  control  over  future  period 

spending  is  possible  through  manipulation  of  ,  s ^  ,  and  x 

max  min  max 

which  is  why  these  parameters  are  included.  Table  5-5  provides  the 
project  success  probabilities  for  the  various  expenditure  levels. ^ 

The  objectives  are: 

1)  Maximize  the  total  anticipated  net  profit. 

2)  Achieve  a  total  expected  output  of  at  least  6.60.^  Total 
expected  output  is  defined  as  the  sum  of  the  probabilities 
of  success  for  each  project  version  selected  over  all  time 
periods . 

3)  Achieve  a  return  on  expenditure  (ratio  of  net  profit  to 

1 8 

total  funds  expended)  of  at  least  45. 

4)  Obtain  a  life  expected  output,  defined  as  the  sum  of  the 
probabilities  of  success  over  all  years,  of  approximately 
1.00  for  project  12,  the  selection  of  which  has  been  pre¬ 
determined  by  top  management. 

To  formulate  this  problem  for  an  application  of  the  thesis  model, 
project  trees  were  developed  as  shown  in  Figure  5-1.  All  of  the  input 


Table  5-4 

Input  Data:  Application 
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Table  5-5 

Project  Success  Probabilities 
for  Various  Expenditure  Levels 
Application  2 

p<*>] 


Expenditure 
Levels  in  $  000 
(x) 

0 

20 

30 

60 

70 

150 

160 

180 

220 

230 

270 


Project  Name 

9  12  16A  16B 


.65 


.42 


.50 


.38 


.80 


.63 


.60 


.72  .60 

.68 


.88 


.912 


■ 


I 
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Figure  5-1 

Project  Trees  for  Thesis  Model 
Application  2 


Project 

Funds 

Funds 

Funds 

Expected 

Expected 

Proje< 

Name 

Yv.  1 

Yr  *  2 

Yr .  3 

Profit 

Output 

Versic 

($000) 

($000) 

($000) 

($000) 

-d —  60 

0 

0 

7,390 

.50 

1 

i - 

»■ —  30 

0 

0 

6,228 

.42 

2 

; —  180 

150 

0 

18,269 

1.35 

3 

: -  180 

0 

150 

17,966 

1.35 

4 

: -  180 

0 

0 

14,940 

.72 

5 

—  150 

180 

0 

18,105 

1.35 

6 

j —  150 

150 

0 

17,479 

1.26 

7 

: —  150 

0 

180 

17,648 

1.35 

8 

_r~ 

; —  150 

0 

150 

17,079 

1.26 

9 

o - 

—  150 

0 

0 

13,080 

.63 

10 

~  0 

180 

150 

16,607 

1.35 

11 

—  0 

180 

0 

13,580 

.72 

12 

[ —  0 

150 

180 

16,457 

1.35 

13 

-  0 

150 

150 

15,889 

1.26 

14 

j -  0 

150 

0 

11,890 

.63 

15 

—  0 

0 

180 

12,345 

.72 

16 

1 -  0 

0 

150 

10,808 

.63 

17 

-  180 

220 

0 

9,144 

1.28 

18 

—  0 

220 

0 

6,661 

.68 

19 

—  220 

180 

0 

9,213 

1.28 

20 

9 - 

-c 

-  180 

180 

0 

8,836 

1.20 

21 

—  0 

180 

0 

5,890 

.60 

22 

-  220 

0 

0 

7,328 

.68 

23 

1 

-  180 

0 

0 

6,480 

.60 

24 

-  160 

160 

0 

1,991 

1.20 

25 

-  60 

160 

0 

1,789 

.98 

26 

! 

- —  0 

160 

0 

1,327 

.60 

27 

i 

-  160 

60 

0 

1,811 

.98 

28 

12 - 

-d 

-  60 

60 

0 

1,510 

.  7  6 

29 

-  0 

60 

0 

878 

•  38 

30 

—  160 

0 

0 

1,460 

.60 

31 

-  60 

0 

0 

966 

.38 

32 

-  270 

270 

0 

23,347 

1.824 

33 

-  230 

270 

0 

23,248 

1.792 

34 

—  0 

270 

0 

19,651 

.912 

35 

16A 

-  270 

230 

0 

23,289 

1.792 

36 

t-, 

-  230 

230 

0 

23,169 

1.76 

37 

-  0 

230 

0 

18,989 

.88 

38 

—  270 

0 

0 

21,618 

.912 

39 

-  230 

0 

0 

20,890 

.88 

40 

16B  — 

— p 

| -  70 

0 

0 

8,570 

.80 

41 

i , 

1 -  20 

0 

0 

7,000 

.65 

42 

j 


J 


16B 


105 


data,  with  the  exception  of  net  discounted  expected  profits,  are 
obtained  directly  from  Tables  5-4  and  5-5,  The  expected  profit 
functions  used  in  Rosen  and  Souder’s  dynamic  programming  model  were 
also  used  to  compute  the  expected  profit  column  entries  for  the 
decision  tree.  For  example,  the  expected  profit  for  version  8,  pro¬ 
ject  6,  is  determined  as  follows.  First,  compute  the  net  expected 
profit  with  one  time  period  remaining: 

f8  =  G8  p8(x)  -  x  =  21,000  (.72)  -  180  -  14,940  (5.1) 

Here,  the  superscript  refers  to  the  projc  version  and  the  sub¬ 
script  refers  to  the  number  of  time  peric.-  ;  emaining.  Next, 
compute  the  net  expected  profit  with  two  time  periods  remaining: 

=  21,000  (0)  -  x  +  . 909 [1  -  0]  (14,940; 

=  13,580.46 


G8  p8(x)  -  x  +  p[l  -  p8 (x) ]  f8 


(5.2) 


Finally,  compute  the  expected  profit  with  three  time  periods 
remaining : 


8  8,  x  r,  8 ,  .  .  _8 

G  p  (x)  -  x  +  p  [1  -  p  ( x)  ]  f  2 


(5.3) 


=  21,000  (.63)  -  150  +  .909  [1  -  .63]  (13,580.46) 

=  17,648 


Uote  that  only  feasible  project  versions  are  included  in  the  decision 
trees.  For  example,  project  6  has  a  three  year  planning  horizon  with 
two  versions  possible  in  each  time  period  giving  a  total  of  27  versions. 
However,  some  of  these  versions  are  infeasible,  such  as  the  version 
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having  funding  levels  of  $  180,000  in  each  year,  since  s  of  $  330,000 

max 

is  exceeded.  The  project  version  labels  in  Figure  5-1  have  been  added 
for  iden t if ication  purposes. 

The  problem  was  solved  using  the  dynamic  programming  recursive 
equations  developed  by  Rosen  and  Souder  with  the  objective  of  maximiz¬ 
ing  net  expected  profit  for  the  given  budget  level  and  other  funding 
restrictions  for  each  project  version.  Then,  using  the  project  versions 
selected  by  the  model,  the  goal  attainment  levels  of  the  other  three 
objectives  were  computed.  Results  are  shown  in  Table  5-6,  column  (1). 
Since  the  return  on  expenditure  objective  was  not  met,  Rosen  and  Souder 
suggest  dropping  out  low  profit  projects,  relaxing  the  funding  level 
constraints  or  relaxing  the  first  or  third  objective  and  rerunning  the 
model.  Here,  the  strategy  they  selected  was  to  relax  the  return  on 
expenditure  goal  and  accept  the  solution  presented  (in  the  original 
problem,  the  return  on  expenditure  goal  was  relaxed  from  55  to  51;  for 
the  smaller  problem  it  is  relaxed  from  45  to  43). 

The  thesis  model  was  also  applied  to  this  problem  by  incorporating 
the  multiple  goals  and  decision  tree  data  into  an  integer  goal  program¬ 
ming  formulation.  The  first  goal  was  to  ensure  that  only  one  or  none 
of  the  versions  of  each  project  is  selected.  The  second  goal  is  to 
stay  within  the  $  750,000  budget  constraint.  The  fourth  goal  was  to 
achieve  the  total  expected  output  of  6.60.  The  fifth  goal  was  to 
achieve  the  life  expected  output  of  approximately  1.00  for  project  12, 
and  the  sixth  goal  was  to  maximize  the  expected  net  profit.  The  only 
objective  that  could  not  be  directly  operationalized  in  the  goal  pro¬ 
gramming  model  was  the  return  on  expenditure  goal  of  45  since  the 


i 


Table  5-6 
Model  Results 
Application  2 


(1) 

(2) 

(3) 

Dynamic 

Thesis 

Thesis 

Programming 

Model 

Model 

Model 

Solution  1 

Solution 

Budget  Level  ($) 

Project  Versions  Selected 

750,000 

750,000 

750,000 

Proj  ec  t  s 

1 

1 

2 

2 

6 

6 

3 

3 

9 

18 

18 

18 

12 

26 

26 

29 

1 6  A 

34 

34 

34 

16B 

41 

41 

41 

Total  Funds  Expended  ($) 

1,580,000 

1,550,000 

1,450,000 

Expected  Net  Profit  ($) 

68 . 246 ,000 

67,248,000 

66,969,000 

Return  on  Expenditure  ($) 

43.19 

43.38 

46.20 

Total  Expected  Output 

6.702 

6.622 

6.40 

Life  Expected  Output  for 
Project  12 


.98 


.76 


computation  requires  a  nonlinear  equation.  However,  this  goal  can  be 
easily  incorporated  by  establishing  a  goal  constraint  for  total  expen¬ 
diture  and  using  several  model  runs  with  incremental  tightening  of  the 
constraint  until  the  return  on  expenditure  goal  is  met.  This  goal  was 
assigned  as  the  third  priority  objective. 

The  thesis  model  was  run  using  the  variable  selection  and  exchange 
heuristics  and  with  partitioning  on  each  project  version.  The  results 
are  shown  in  colums  (2)  and  (3)  of  Table  5-6.  Solution  1  shown  in 
column  (2)  involved  a  very  loose  constraint  on  the  third  priority 
goal  (incremental  ranges  from  $  1,580,000  to  $  1,600,000)  and  the 
results  reflect  slight  but  interesting  differences  when  compared  to 
the  dynamic  programming  solution.  The  expected  net  profit  is  1.52 
less  than  the  Rosen  and  Souder  solution  but  there  is  a  slight  improve¬ 
ment  (.4%)  on  the  return  on  expenditure  goal  attainment.  Both  solu¬ 
tions  attain  the  total  expected  output  goal.  Rather  than  changing 
the  return  on  expenditure  goal  to  43  and  accepting  this  solution,  the 
model  was  rerun  at  tighter  total  expenditure  levels  until  the  return 
on  expenditure  exceeded  the  goal  of  45.  This  goal  was  achieved  with 
the  projects  selected  and  shown  under  solution  2,  column  (3)  in 
Table  5-6.  The  expected  net  profit  is  1.92  less  than  Rosen  and 
Souder ?s  solution  but  the  return  on  expenditure  goal  is  now  achieved. 
Note  that  the  total  expected  output  and  the  project  12  life  expected 
output  goals  are  underattained  in  this  solution.  The  two  solutions 
presented  (for  the  thesis  model)  are  an  example  of  the  decision 
aiding  capability  of  the  thesis  model,  ttodel  users  can  readily 
determine  that  the  simultaneous  attainment  of  the  four  goals  is  not 
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possible  and  the  decision  to  be  made  is  to  accept  the  strategy  pro¬ 
viding  a  higher  expected  profit  with  a  lower  return  and  higher  expected 
successes  or  to  accept  a  strategy  with  the  reverse  results. 

Application  3:  A  Static  Multiple  Objective  (Zero-One  Goal 
Programming)  Case 

The  final  efficiency  and  validity  evaluation  of  the  model  was 

performed  for  a  static  multiple  objective  R&P  project  selection 

problem.  The  model  chosen  for  comparison  purposes  was  the  zero-one 

goal  programming  model  presented  recently  by  Keown,  Taylor,  and 
19 

Duncan,  hereafter  referred  to  as  the  KTD  model.  These  researchers 

20 

used  the  zero-one  implicit  enumeration  algorithm  developed  by  Morris 
and  previously  discussed  in  Chapter  III  and,  because  of  the  recent 
vintage  which  the  model  represents,  for  all  intents  and  purposes,  the 
current  state-of-the-art  in  multiple  objective  R&D  project  selection 
models. 

The  problem  used  for  evaluation  of  the  two  models  is  the  20 
project  hypothetical  case  study  Keown,  Taylor  and  Duncan  used  in 
illustrating  their  approach.  Input  data  is  shown  in  Table  5-7.  The 
study  concerns  an  electrical  equipment  manufacturer  involved  in 
offensive  (new  product  development)  and  defensive  (modification  of 
existing  products)  projects,  and  having  two  sales  divisions,  two 
competitive  markets  and  two  research  centers.  To  choose  a  portfolio 
strategy  for  the  next  planning  period,  the  company  has  established  the 
following  10  goals: 

1)  Avoid  project  duplication  by  selecting  a  maximum  of  one 
version  from  each  of  the  following  project  combinations: 


Table  5-7 

Input  Data  for  Application 
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projects  1  and  2,  7  and  8,  9  and  10,  and  19  and  20  (these 
are  commonly  referred  to  as  mutually  exclusive  system 
constraints) . 

2)  Do  not  exceed  an  expenditure  ceiling  of  $  2,100,000. 

3)  Do  not  exceed  the  capacity  of  the  two  research  facilities. 

4)  Do  not  exceed  a  staff  requirement  of  more  than  22 
researchers  at  Center  1  and  25  researchers  at  Center  2. 

5)  Select  at  least  two  projects  representing  military 
applications. 

6)  Select  at  least  three  offensive  and  five  defensive  projects 
for  the  final  portfolio. 

7)  Achieve  a  total  expected  output  of  at  least  2.00  (sum  of 
the  probabilities  of  success  for  projects  selected). 

8)  Achieve  an  expected  annual  sales  increase  of  $  750,000  in 
Division  1  and  $  500,000  in  Division  2. 

9)  Achieve  an  expected  increase  in  market  share  of  2.80%  in 
Market  1  and  2.20%  in  Market  2. 

10)  Maximize  the  net  present  value  of  the  projects  selected. 

The  problem  was  solved  using  the  KTD  model  (Morris  algorithm)  and 
three  versions  of  the  thesis  model.  The  results  are  reported  in  Table 
5-8.  For  the  thesis  model,  solution  1  represents  the  results  for  the 
variable  selection  and  exchange  heuristic.  Solution  2  results  were 
obtained  by  including  the  partitioning  procedure  where  each  mutually 
exclusive  project  version  was  iteratively  forced  out  of  consideration 
for  the  solution.  Solution  3  results  were  obtained  by  creating  20 
subproblems  for  the  partitioning  routine,  where  each  subproblem  repre¬ 
sents  one  project  forced  out  of  consideration  for  the  solution.  From 


Table  5-8 
Model  Comparisons 
Application  3 
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CPU  Time  (Seconds) 
Iterations 
Projects  Selected 


Goal  Achievements 

1*  Meet  System 
Constraints 

2.  Budget  Limit 

3.  Facilities 
Capacity 

4.  Staff  Limit 

5.  Priority  Pro¬ 
jects  Goal 

6.  Offensive- 
Defensive 
Balance 

7.  Risk  Spread¬ 
ing  Goal 

8.  Sales  Expected 

($) 

9.  Market  Share 
Expected 

10.  Net  Present 

Value  Expected 

($) 


KTD  Model 

Thesis 

Model 

Soln  1 

Thesis 

Model 

Soln  2 

Thesis 

Model 

Soln  3 

1202.0 

1.3 

5.9 

14.0 

53,472 

26 

208 

499 

1-4-6-8-9 

12-18-20 

1-5-6-8 

13-14-15 

16 

1-3-6-8 

14-15-16 

18 

1-4-6-8-9 

14-15-16 

13 

Attained 

Attained 

Attained 

Attained 

11 

11 

M 

11 

tl 

II 

II 

ff 

II 

11 

II 

II 

II 

11 

If 

II 

11 

II 

11 

M 

II 

11 

M 

ir 

1,077,600 

1,010,480 

1,055,420 

1,062,320 

4.599% 

4.618% 

3. 758/. 

3.668% 

860,000 

780,000 

850,000 

860,000 

an  efficiency  standpoint,  the  thesis  model  is  clearly  superior  with  CPU 
times  ranging  from  1.3  to  14  seconds  compared  to  "just  over  20  minutes 
for  the  KTD  model.  It  is  noted  again  that  20  variables  is  the  approx¬ 
imate  upper  limit  of  the  capability  of  the  Morris  algorithm.  With 
respect  to  solution  validity,  the  thesis  model  was  very  close  to  the 
optimal  solution  (KTD  model)  for  all  versions  used.  The  first  seven 
goals  were  completely  attained  by  the  thesis  models  as  they  were  in 
the  optimal  solution.  For  the  KTD  model,  the  sales  goal  attainment 
level  was  86.2 %  of  the  stated  objective.  Solution  1  of  the  thesis 
model  attained  80.8a  of  this  goal,  solution  2  attained  84.471  and  solu¬ 
tion  3  attained  85%  of  this  goal.  The  market  share  growth  goal  was 
not  attained  by  any  of  the  models.  The  percent  attainment  levels  were 
92%  for  the  KTD  model  and  92.4%,  75.7%  and  73.4%  for  the  three  versions 
of  the  thesis  model.  The  net  present  value  goal  was  simply  to  maximize 
the  net  present  value  of  the  portfolio.  The  three  versions  of  the 
thesis  model  attained  90.77,  98.8%  and  100%  of  the  optimum  goal  attain¬ 
ment  level  of  $  860,000. 

Summary  of  the  Efficiency  and  Validity  Comparisons 

The  first  application  involved  comparing  the  thesis  model  with  the 
integer  (zero-one)  programming  algorithm  of  Mandakovic  for  the  static 
single  objective  case.  The  thesis  model  outperformed  Mandakovic's 
model  with  respect  to  solution  results  whereas  the  reverse  was  true  with 
respect  to  program  processing  times.  For  very  large  problems  of  this 
type,  that  is,  1,000  to  10,000  variables,  the  Mandakovic  model  is 
obviously  superior.  However,  two  important  points  must  be  made.  First, 
the  thesis  model  was  not  designed  for  situations  where  the  R&D  manager 
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must  gather  data  for  and  select  projects  from  a  set  of  10,000  project 
variables,  or  even  1,000  project  versions.  It  was  designed  for  medium 
scale  problems  with  an  upper  limit  of  90  to  100  variables.  While  the 
large  scale  problems  have  theoretical  interest,  the  medium  scale 
problems  represent  the  realistic  situations  found  in  the  literature 
and  during  the  interview  sessions  with  R&D  laboratory  directors. 
Secondly,  the  richness  of  the  thesis  model  lies  in  its  capability  to 
address  multiple,  noncommensurate  objectives  ranked  on  an  ordinal 
scale;  a  capability  which  cannot  be  evaluated  in  this  application  due 
to  the  single  criterion  restriction  of  Mandakovic's  model. 

The  second  application  compared  the  thesis  model  to  Rosen  and 
Souder’s  dynamic  programming  model  for  the  multiperiod,  multiple 
objective  case.  The  solution  generated  by  the  thesis  model  was 
comparable  to  the  dynamic  programming  solution.  To  complete  the 
comparison  of  the  two  models,  it  is  helpful  to  point  out  the  advan¬ 
tages  and  disadvantages  of  both  approaches.  The  thesis  model  can 
incorporate  all  objectives  into  the  problem  formulation  (except  the 
nonlinear  return  on  expenditure  function  which  could  be  indirectly 
handled),  whereas  the  dynamic  programming  approach  cannot.  The  dynamic 
programming  model  can  handle  a  large  number  of  project  variables, 
whereas  the  thesis  model  was  designed  for  medium  size  problems.  On 
the  other  hand,  the  thesis  model  can  handle  multiple  resource  con¬ 
straints  over  multiple  periods  whereas  the  dynamic  programming  model 
5  is  limited  to  a  single  constraint  in  the  first  stage.  In  summary,  for 

medium  size  problems  the  thesis  model  can  be  used  to  solve  any  R&D 
t  project  selection  problem  solved  by  the  dynamic  programming  model. 


The  converse  is  not  true  since  the  dynamic  programming  technique  is 
restricted  to  a  single  constraint  and  single  criteria. 
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The  third  application  compared  the  thesis  model  with  the  zero-one 
goal  programming  algorithm  by  Morris  as  applied  by  Keown,  Taylor  and 
Duncan  for  a  single  period,  multiple  objective  problem.  With  respect 
to  solution  validity,  the  thesis  model  was  very  close  to  the  optimal 
solution  and  the  processing  times  of  1.3  to  14  seconds  represented  a 
substantial  reduction  from  the  20  minutes  required  by  the  Morris 
algorithm.  Further,  it  was  pointed  out  that  the  problem  solved  by 
the  Morris  algorithm  represented  its*  maximum  size  capability,  whereas 
the  thesis  model  can  solve  problems  with  substantially  more  decision 
variables.  Advantages,  in  addition  to  the  speed  and  problem  size 

aspects,  of  the  thesis  model  in  comparison  with  the  KTD  model  include  f 

the  capability  to  address  multiple  time  periods  characterized  by 

uncertainty. 

Practical  Utility  of  the  Thesis  Model 
As  Viewed  Through  Interviews  with  R&D  Managers 

Interviews  were  conducted  with  R&D  managers  in  five  enterprises 
having  professionally  staffed  and  formally  organized  research  and 
development  activities.  The  enterprises  were  chosen  to  be  representa¬ 
tive  of  a  diversity  of  organizat ional  types  with  supporting  R&D  lab¬ 
oratories,  concentrating  on  a  variety  of  programs  ranging  from  explor¬ 
atory  research  for  the  advancement  of  knowledge  to  development  projects 
for  new  products  or  processes.  A  brief  description  of  the  sample 
laboratories  and  parent  organizations  is  provided  on  the  following 


page. 
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Case  1:  A  research  laboratory  of  a  large  textile  corporation 

engaged  in  applied  research  on  domestic  textiles. 

Case  2:  The  research  and  development  laboratories  of  a  major 

pharmaceutical,  biological  and  chemical  producer  which 
concentrates  on  exploratory  research  programs. 

Case  3:  A  research  laboratory  of  a  government  regulatory 

agency  involved  in  applied  research,  development  of 
standards  and  control  technology. 

Case  4:  The  chemical  research  division  of  a  major  consumer 

products  company  engaged  in  chemical,  physical  and 
analytical  studies,  and  in  applied  research  on  new 
products  and  manufacturing  processes. 

Case  3:  A  center  for  technology  applications  of  an  independent, 

no t-for-prof i t  research  institute  involved  in  basic  and 
applied  research  in  medical  and  related  sciences. 
Research  work  is  conducted  on  a  fee  or  contract  basis 
with  government,  industry  and  foundation  organizations. 

The  interview  sessions  involved  presentation  of  the  multiple 
objective  project  selection  and  resource  allocation  model  followed  by 
discussions  of  the  R&D  decision  making  processes  unique  to  the  five 
organizations.  Specifically,  the  nature  of  the  project  selection 
decision  was  examined  with  regards  to  the  realism,  applicability, 
feasibility  and  limitations  of  the  thesis  model.  A  summary  of  the 
findings  for  each  of  the  five  organizations  is  described  next. 

Case  1 

The  director  of  research  for  the  textile  corporation  has  a  staff 
of  25  personnel,  including  10  professionals,  and  is  responsible  to  the 


vice  president  of  technical  services.  Research  work  is  concentrated 
in  developing  methods  for  preparation,  dyeing  or  finishing  new  fiber 
or  fabric  textile  products.  Methods  developed  are  then  transferred  to 
the  quality  control  department  where  pilot  tests  for  tensile  strength, 
shrinkage,  flammability,  fading  and  other  characteristics  are  made. 

Once  tested,  the  research  projects  are  transferred  to  the  product 
development  department  for  final  development,  design  efforts,  and 
pilot  mill  runs  which  precede  product  production  at  one  of  the  firm’s 
mills. 

Ideas  for  new  fibers,  fabrics,  finshes  and  production  methods 
originate  from  a  number  of  sources,  the  most  important  of  which  are 
the  research  laboratory,  product  development  and  design  departments, 
corporate  management,  and  the  marketing  group.  These  ideas  or  project 
proposals  are  presented  by  the  directors  of  research,  marketing  and 
development  departments  at  bi-annual  corporate  product  development 
meetings  for  group  evaluations.  New  projects  selected  are  incorporated 
with  the  set  of  approved  active  projects  and  the  six  month  corporate 
research  and  project  development  plan  is  finalized.  Most  projects  have 
a  short  time  horizon  with  the  time  from  initiation  of  research  to  mill 
production  typically  being  two  years.  The  evaluation  process  normally 
includes  consideration  of  approximately  30  new  and  existing  textile 
and  production  method  improvement  projects.  Formal,  quantitative 
selection  models  are  not  being  used  as  decision  aiding  tools. 

The  director  of  research  was  very  interested  in  the  proposed 
multistage,  multiple  objective  model  and  gave  indications  that  the 
thesis  model  is  applicable  in  their  planning  processes.  Project 
proposal  data  is  available  and  includes  estimates  of  technical  success, 
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cost  data,  and  budget  and  personnel  resources  required.  The  decision 
tree  format  was  viewed  as  a  workable  planning  concept.  Multiple  goals 
for  the  research  effort  are  definitely  in  existence  and  the  model’s 
capability  for  handling  multiple  criteria  became  a  major  selling  point. 
Further,  the  firm’s  goals  are  considered  in  an  ordinal  type  ranking 
and,  during  the  course  of  the  interview,  the  research  director  pro¬ 
vided  the  following  exmaple  of  the  goal  set  : 

Priority  1:  Select  those  projects  that  are  compatible  with  the 
firm’s  long  range  aggregate  product  run  plans. 

Priority  2:  Select  those  projects  that  maximize  the  probability 
of  technical  successes. 

Priority  3:  Select  those  projects  that  minimize  chemical  input 
and  manufacturing  costs. 

Priority  4:  Select  those  projects  that  satisfy  an  urgency  of 
need  cirteria. 

The  research  director  noted  that  the  priority  structure  changes  over 
time  and  sensitivity  analysis  on  various  goal  priorities  is  a 
required  capability.  It  is  also  noted  that  the  multiple  goals 
represent  an  interesting  situation  since  they  are  noncommensurate 
and  conflicting  in  nature. 

An  additional  area  discussed  for  study  within  the  model  framework 
is  the  critical  manpower  shortage  problem  generated  by  the  requirement 
placed  on  the  research  director  to  provide  technical  services  to  the 
firm’s  mills.  Here,  sensitivity  analysis  may  be  used  to  analyze  "what 
if"  questions  concerning  the  research  program  that  could  be  conducted 
if  this  task  was  transferred  to  another  department  ot  staff  agency. 
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Model  results  would  be  used  to  demonstrate  to  top  management  the  need 
to  split-off  technical  services. 

In  summary,  the  research  director  was  enthusiastic  about  appli¬ 
cation  of  the  multiple  objective  model  which  is  commensurate  with  his 
needs,  and  is  a  realistic  concep tual izat ion  of  the  decision  making 
process.  As  a  result,  an  actual  application  of  the  thesis  model  was 
developed  and  is  reported  on  in  the  final  section  of  this  chapter. 

Case  2 

The  research  laboratories  of  the  pharmaceutical  firm  are 
principally  engaged  in  exploratory  research  with  regards  to  chemo¬ 
therapy  of  microbial  infections,  antitumor  agents,  immunosuppressives, 
agents  for  cardiovascular  disease  and  drug  interac t ions .  Other 
activities  include  pharmaceut ical ,  agriculture  medicine  and  veterinary 
medicine  product  support;  and  applied  product  research  and  development. 
Profits  generated  by  products  marketed  by  the  pharmaceut ical  company 
are  channelled  back  to  the  research  laboratories  and  to  the  parent 
organization  which  supports  other  subsidiaries  and  medicine  research. 
The  interview  was  conducted  with  the  director  of  research,  development 
and  medicine  (RDM)  administration  of  the  research  laboratories  con¬ 
sisting  of  more  than  700  personnel. 

The  R&D  project  selection  decision  is  difficult  to  clearly  define 
in  this  organization  because  of  the  concentration  on  exploratory 
research  where  the  emphasis  is  centered  on  the  advancement  of  medicine 
knowledge  with  no  specific  end  product  as  the  objective.  However, 
since  some  of  the  exploratory  research  has  generated  marketable 
pharmaceutical  products,  it  was  of  Interest  to  further  examine  project 


evolution  in  these  instances.  Exploratory  research  is  conducted  in 


the  organic  chemistry  department.  Here,  scientists  study  new  com¬ 
pounds  having  desirable  biological  activity.  Once  synthesized,  the 
compounds  go  to  the  pharmacology  department  where  therapeutic  useful¬ 
ness  is  tested.  The  next  step  requires  compound  testing  in  the 
experimental  therapy  department.  If  promising,  a  research  program 
is  further  defined  into  a  project  which  is  presented  to  a  research 
committee  by  a  project  champion.  The  committee  subjectively  evaluates 
the  proposal  and  makes  a  go/no  go  decision.  If  approved  for  further 
work,  a  project  team  is  formed  for  product  development  in  the  medicine 
division  and  chemical  development  laboratories.  There  are  approximately 
30  development  projects  on  going  or  under  evaluation  at  any  one  time. 

While  it  is  not  uncommon  for  research  laboratories  engaged  in 
exploratory  research  to  not  u^e  formal  selection  models  for  project 
selection  nor  gather  quantitative  data  for  proposal  evaluation,  the 
RDM  administrator  was  nevertheless  excited  about  the  thesis  model  and 
requested  that  I  repeat  my  presentation  to  a  representative  from  the 
company’s  statistics  department.  Apparently  the  parent  organization 
recently  requested  that  all  laboratories  become  involved  in  project 
accounting  and  budget  planning;  areas  which  have  received  no  attention 
in  past  years.  As  a  first  step  in  response  to  this  directive,  the 
research  committee  implemented  a  simple  scoring  type  model  to  evaluate 
technical  success  probabilities  for  development  projects  currently 
underway , 

The  RDM  administrator  indicated  that  the  multiple  objective 
model  is  feasible,  useful  and  realistic  for  laboratories  principally 


engaged  in  selection  of  development  projects,  but  perhaps  less 


useful  for  selection  of  exploratory  research  efforts.  In  this  par¬ 
ticular  situation,  it  was  noted  that  before  a  commitment  is  made  to 
utilize  the  thesis  model  for  development  project  selection  and 
resource  allocation,  a  commitment  must  be  made  to  a  data  gathering 
and  information  system. 

Case  3 

The  research  laboratory  of  a  government  regulatory  agency 
concentrates  on  applied  in-house  and  contracted  research  programs 
to  generate  information  concerning  the  health  effects  of  a  wide 
variety  of  chemicals  and  drugs  for  the  development  of  environmental 
standards  and  control  technology.  The  manager  interviewed  was  the 
deputy  director  of  the  research  laboratory  employing  approximately 
350  professionals. 

Although  the  deputy  director  showed  interest  in  the  multiple 
objective  project  selection  model,  he  believed  that  the  model  is 
better  suited  for  project  termination  rather  than  selection  in 
their  environment.  The  research  laboratory  supports  a  regulatory 
agency  and,  as  such,  the  majority  of  the  project  selection  decisions 
are  mandated;  that  is,  the  project  selection  problem  occurs  outside 
of  the  laboratory.  For  example,  when  General  Motors  announced 
plans  for  widespread  introduction  of  diesel  powered  passenger  cars 
in  the  early  1980* s.  Congress  passed  a  law  directing  the  regulatory 
agency  to  develop  emission  standards  and  control  technology.  The 
research  laboratory  was  given  the  task  of  discovering  the  health 
effects  of  various  levels  of  atmospheric  particulates.  Although  some 
latitude  is  available  in  selecting  the  major  method  used  for  the 
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study,  time  and  cost  ruled  out  epidemology  and  clinical  studies  and, 
as  a  result,  animal  toxicology  was  selected  as  the  only  feasible 
study  alternative.  Even  though  a  wide  variety  of  animal  toxicology 
studies  are  available,  including  bioassay,  study  of  carcinogens, 
inhalent  studies  and  skin  painting,  the  method  selection  problem  did 
not  appear  significant.  Further,  many  of  the  mandated  projects  repre¬ 
sent  medium  to  long  range  time  commitments.  As  such,  the  research 
portfolio  for  the  next  year  includes  the  same  projects  being  funded 
this  year  with  no  new  proposals  added  to  the  portfolio. 

Project  termination,  however,  is  distinctly  the  responsibility 
of  the  laboratory  directors.  For  example,  the  research  organization 
received  a  sizable  budget  cut  during  the  course  of  a  recent  fiscal 
year.  It  followed  that  a  number  of  projects  had  to  be  terminated 
before  their  completion  in  order  to  stay  within  the  revised  budget. 

The  project  directors  were  asked  to  rank  order  assigned  projects. 

The  laboratory  directors  then  eliminated  a  set  of  the  lowest  priority 
projects  with  a  research  dollar  amount  equivalent  to  the  budget  cut. 

A  subjective  evaluation  procedure  was  used  for  the  termination  deci¬ 
sion,  implicitly  considering  multiple  objectives  (one  of  the  goals  was 
to  maintain  in-house  work  and  terminate  contracted  projects).  It  was 
noted  that  the  thesis  model  is  feasible  for  this  situation  as  project 
priorities  can  be  represented  in  benefit  functions  for  one  of  the 
multiple  goals  and  several  mode]  runs  can  he  made  to  represent  various 
funding  levels. 

The  deputy  director  did  indicate  that  the  multiple  objective 
model  is  feasible  for  research  laboratories  where  selection  decisions 
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for  research  programs  are  made  rather  than  mandated  by  higher 
authority.  The  example  problem  presented  was  useful  in  describing 
these  types  of  realistic  situations. 

Case  4 

The  manager  of  the  research  division  of  a  major  consumer  products 
company  is  responsible  to  the  director  of  research  and  supervises  a 
staff  of  60  personnel,  including  20  doctorates.  The  research  division 
concentrates  in  applied  research  tied  to  development  of  nets’  products 
and  processes  and,  together  with  the  analytical  and  science  information 
divisions,  works  on  chemical,  physical  and  analytical  studies  of  the 
firm’s  products.  The  research  division  also  works  closely  with  the 
development  department,  which  is  responsible  for  product  and  process 
development,  and  with  the  marketing  research  department.  Although  the 
industry  is  classified  as  one  of  low  technology,  the  research  and 
development  activity  is  intense  with  over  530  personel,  including  260 
professionals  (40  doctorates),  actively  engaged  in  R&D. 

In  the  early  1970’s,  research  project  selection  followed  the 
general  descriptive  study  as  presented  in  Chapter  I.  Between  40  and 
50  project  proposals  were  generated  each  year  and  data  profiles  were 
carefully  put  together.  Projects  were  selected  using  subjective 
evaluation  processes  taking  into  account  the  ranked  multiple  objec¬ 
tives  identified  in  a  corporate  MBO  plan.  Currently,  the  research 
division  has  moved  away  from  project  selection  as  a  result  of  imple¬ 
mentation  of  a  corporate  wide  performance  management  planning  system 
which  is  operationalized  in  annual  corporate  working  plans.  All 
divisions,  including  research,  identify  key  areas  of  concentration 


for  the  next  year.  The  corporate  planning  department  develops  the 
annual  plan  from  these  areas,  tying  in  the  corporate  goals  and 
marketing  strategies.  The  annual  plan  then  becomes  a  working 
plan  for  research  divisions.  The  working  plan  contains  broad  based 
funded  areas  of  research  as  opposed  to  specific  projects,  and  this 
constitutes  fifty  percent  of  the  level  of  effort  of  the  research 
staff.  The  remaining  research  capability  is  devoted  to  technical 
services  support  requirements  and  anticipatory  research,  where  project 
selection  is  either  mandated  or  otherwise  inappropriate. 

The  research  director,  based  on  his  previous  experience  with 
project  selection  schemes,  viewed  the  proposed  multiple  objective 
model  as  feasible  in  organizations  having  formal  project  selection 
processes  and  noted  that  the  consideration  of  multiple  objectives 
and  uncertainty  were  realistic  characteristics.  The  director  did 
state  that  under  the  former  system  of  project  selection,  the  multiple 
objective  model  could  have  been  a  workable  decision  aiding  tool.  At 
present,  however,  the  model  would  be  more  useful  for  their  analytical 
chemistry  group  where  more  defined  selection  and  allocation  decisions 
are  made . 

Case  5 

The  independent,  not-for-profit  research  institute  is  engaged  in 
basic  and  applied  research  in  a  wide  variety  of  areas  such  as  pharma¬ 
cology,  polymer  science,  environmental  chemistry,  biomedical  engineer¬ 
ing,  atmospheric  chemistry,  and  space  systems.  The  total  research 
staff  numbers  more  than  400  professionals  and  340  technicians  and 
auxiliaries.  The  senior  official  interviewed  was  the  Director  of 


the  Center  for  Applied  Technology. 


Research  capability  is  maintained  and  expanded  through  revenue 
generated  by  research  projects  conducted  for  government,  industry  and 
foundation  organizations  on  a  contract  basis.  Approximately  eighty 
percent  of  the  research  staff  is  allocated  to  this  work.  The  remain¬ 
ing  staff  is  allocated  to  program  development  which  is  a  marketing 
function  involved  with  finding  clients  for  future  research  projects 
and  developing  contract  proposals.  Accordingly,  in  this  unique 
research  setting,  project  selection  is  accomplished  by  the  clients 
and  the  organization  attempts  to  match  its  capability  with  the  needs 
of  the  clients.  The  only  selection  procedure  envisioned  by  the 
center  director,  occurs  at  the  annual  meeting  of  the  institute’s  vice 
presidents  where  areas  of  research  concentration  are  identified 
through  management  consensus. 

Although  the  multiple  objective  project  selection  model  is  not 
applicable  for  this  institute,  the  center  director,  who  has  recently 
developed  a  project  selection  model  for  a  government  agency,  gave  an 
encouraging  assessment  and  believed  the  model  to  be  realistic  and 
feasible  for  those  research  activities  faced  with  the  problem  of 
project  selection. 

Summary  of  the  Findings  of  the  Interview  Sessions 

The  survey  of  five  organizations  involved  in  R&D  activities, 
which  was  carried  out  as  part  of  this  study,  found  agreement  with 
the  findings  of  other  researchers  that  quantitative  project  selection 
models  are  not  being  used  in  the  field.  As  noted  in  Chapter  I,  the 
most  significant  reason  for  this  lack  of  application  is  failure  of 
the  model  builders  to  account  for  realism  with  respect  to  multiple 
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objectives,  multiple  stages  in  project  evolution  and  uncertainty. 
However,  three  additional  reasons  were  identified  during  the  inter¬ 
view  sessions.  First,  in  the  case  of  at  least  two  of  the  firms, 
there  is  a  lack  of  awareness  of  management  science  models  proposed 
as  decision  aiding  tools.  Second,  especially  in  the  case  of  the 
pharmaceutical  company,  project  proposal  data  is  not  collected  at  a 
level  necessary  for  use  of  quantitative  selection  models.  Third, 
in  the  case  of  the  research  institute  and  government  agency,  project 
selection  occurs,  to  some  extent,  at  the  client  or  sponsoring  activity 
level  rather  than  within  the  research  organization  and  the  R&D 
activity  responds  with  its  capability.  These  and  other  findings  con¬ 
cerning  the  potential  for  application  of  the  multiple  objective  model 
are  summarized  in  Table  5-9. 

In  summary,  there  was  a  high  level  of  interest  shown  by  all  of 
the  R&D  managers  and  directors  to  the  proposed  model.  A  consensus 
opinion  appeared  to  be  that  the  multistage,  multiple  objective  model 
is  most  applicable  for  those  R&D  activities  concentrating  on  applied 
research  or  development  projects  which  are  associated  with  clearly 
defined  end  uses  and  where  a  certain  degree  of  autonomy  exists  with 
regards  to  selecting  the  research  portfolio.  The  capability  to 
address  multiple  objectives,  project  evolution  over  the  multistage 
horizon,  and  technical  (internal)  and  outcome  (external)  uncertainty 
was  judged  a  valid  attempt  to  incorporate  the  realism  of  the  research 
environment.  The  example  problem  illustrated  through  decision  tree 
project  planning  and  multiple  objective  formulations  was  received  as 
representative  of  the  type  of  situation  occurring  in  the  research 


Table  5-9 

Summary  of  Survey  Findings 


basis 


activities  faced  with  a  selection  problem.  In  summary,  the  multiple 
objective  model  appears  to  be  a  feasible  approach  which  incorporates 
realistic  characterist ics  of  the  selection  process. 

Application  of  the  Thesis  Model 
In  A  Corporate  Setting 

The  fourth  and  final  examination  of  the  practical  utility  of 
the  thesis  model  concerns  an  application  involving  R&D  project 
selection  and  resource  planning  in  the  research  department  of  a 
large  textile  firm.  A  general  description  of  the  research  organiza¬ 
tion  and  the  R&D  decision  making  processes  within  this  firm  was 
provided  in  the  summary  of  the  interview  sessions  reported  on  pages 
116-119.  It  was  noted  that  the  director  of  research  was  enthusiastic 
about  incorporating  the  thesis  model  as  a  decision  aiding  tool  for 
project  selection  and  resource  planning,  and  the  results  reported 
here  represent  the  inaugural  application. 

The  current  problem  of  interest  to  the  director  of  research 
concerns  the  selection  of  new  research  projects  for  the  next  four 
quarters  of  the  planning  horizon.  An  additional  issue  to  be 
addressed  as  part  of  the  selection  problem  concerns  the  shortage  of 
professional  staff  available  for  research.  Currently,  the  research 
director  is  responsible  for  providing  the  firm's  textile  mills  with 
on-site  technical  support  services.  If  this  responsibil ity  could  be 
split-off  to  another  department,  or  decentralized  through  an  intensive 
training  programming  of  staff  personnel  at  the  mill  locations,  more 
resources  would  be  available  resulting  in  the  possible  expansion  of 
the  portfolio  of  research  projects.  Thus,  it  was  desired  that  the 
thesis  model  be  used  for  the  recommendation  of  two  portfolios;  one, 
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given  the  departmental  responsibility  for  providing  technical  services, 
and  the  other,  considering  the  relief  of  this  responsibility.  In  both 
cases,  the  portfolio  is  to  be  selected  so  as  to  best  satisfy  a  set  of 
miltiple  objectives  for  the  firm’s  research  effort. 

A  detailed  description  of  the  problem  begins  with  the  project 
proposal  decision  trees  shown  in  Figure  5-2.  The  critical  and  con¬ 
straining  resource,  professional  staff  available  for  research,  is 
shown  at  the  bottom  of  the  illustration  for  each  of  the  four,  three 
month  planning  periods,  both  with  and  without  the  technical  support 
responsibility.  Two  main  points  are  noted  to  clarify  the  numbers 
provided.  First,  the  fluctuations  in  staff  availability  over  the 
planning  horizon  result  from  the  seasonal  nature  of  product  line 
development  in  the  textile  industry.  During  certain  quarters  of  the 
year,  some  of  the  research  staff  is  assigned  to  product  line  develop¬ 
ment  activity  which  results  in  less  staff  available  for  research.  In 
Figure  5-2,  this  additional  activity  is  reflected  in  the  first  and 
fourth  quarters.  Second,  it  is  shown  that  about  half  of  the  research 
staff  is  committed  to  technical  support  services,  which  is  a  commitment 
ratio  based  upon  the  most  recent  historical  information.  Thus,  out 
of  the  eight  or  nine  pro fessionals  available  in  the  research  department, 
depending  on  the  quarter  of  the  year,  four  are  programmed  for  technical 
services.  It  was,  however,  indicated  by  the  research  director  that 
general  business  conditions  can  effect  this  ratio;  ’’when  business  is 
good,  technical  assistance  declines;  when  business  gets  slack,  the  mills 
ask  for  more  assistance  with  their  problems."  The  decision  trees  also 
reflect  the  staff  resources  required  for  each  project  during  each 
quarter  and,  in  the  case  of  projects  B  and  E,  the  staff  required  for 


Figure  5-2 

Application  4:  Project  Trees 
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each  mutually  exclusive  project  version. 

Columns  6  through  9  of  the  decision  trees  provide  information  that 

link  each  of  the  proposed  projects  to  the  multiple  objectives  for  the 

firm’s  research  effort.  Column  6  reflects  whether  or  not  the  proposed 

project  is  compatible  with  long  range  product  development  plans.  Column 

7  provides  a  success  index  which  ranges  over  the  integers  1  through  3, 

where  1  indicates  a  low  probability  of  success  and  3  indicates  a  high 

probability.  Column  8  includes  the  chemical  input  and  manuf ac t ur ing 

cost  indices  that  also  range  over  the  integers  1  through  3,  with  1 

representing  a  low  cost  range  and  3  a  high  cost  range.  Column  9 

21 

reflects  whether  or  not  a  proposed  project  is  urgently  needed. 

Using  the  information  and  relationships  provided  in  the  decision 
trees,  the  next  step  involved  the  formulation  of  the  zero-one  goal 
programming  models.  Two  models  were  developed  for  the  initial 
analysis  of  the  problem.  Model  I  considers  the  case  where  technical 
services  are  the  responsibility  of  the  research  department  and  Model  II 
considers  that  technical  services  are  not  the  department’s  responsi¬ 
bility.  The  multiple  objectives  and  related  constraint  equations, 
which  are  given  in  Table  5-10,  are  described  below.  Unless  specifically 
noted,  each  priority  goal  and  constraint  equation  is  common  to  the 
two  models. 

Priority  1:  Ensure  that  only  one  of  the  mutually  exclusive 
versions  of  projects  B  and  E  is  selected  if  either  or  both  of  the 
projects  is  included  in  the  final  portfolio.  As  in  other  applications 
involving  mutually  exclusive  system  constraints,  this  goal  must  be 
satisfied  to  prevent  the  duplication  of  effort  that  would  result  by 
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accepting  two  or  more  versions  of  the  same  project.  This  goal  is 
satisfied  by  minimizing  the  positive  deviational  variables,  p^  and  p^  , 
in  rows  1  and  2  of  Table  5-10. 

Priority  2:  Minimize  overruns  in  staff  requirements  for  the 

selected  portfolios.  For  Model  I,  this  goal  is  achieved  by  minimizing 

the  positive  deviational  variables,  p  through  p  ,  in  rows  3-6.  Note 

3  6 

that  each  row  represents  one  period  in  the  planning  horizon.  For 
Model  II,  the  goal  is  satisfied  through  minimization  of  p^  through  p 
in  rows  7-10. 

Priority  3:  Select  those  projects  that  are  compatible  with  the 

firm’s  long  range  product  development  plans.  This  goal  is  attained  by 

22 

minimizing  p^  from  a  right  hand  side  of  zero  in  row  11. 

Priority  A:  Maximize  the  index  of  technical  successes.  The 

constraint  equation  for  this  goal,  shown  in  row  12,  is  formulated  with 

a  linear  combination  of  project  variables  whose  coefficients  are  taken 

from  column  7  of  Figure  5-2.  By  minimizing  the  negative  deviational 

variable,  n^>  fro™  a  large  right  hand  side  value,  this  goal  can  be 

attained  since  the  zero-one  goal  program  will  attempt  to  select  those 

projects  having  high  probability  of  success  indices. 

Priority  5:  Minimize  chemical  input  and  manufacturing  process 

indices.  The  constraint  equations  for  this  goal,  included  in  row  13 

(chemical  input  costs)  and  row  14  (manuf ac t ur ing  process  costs)  involve 

a  linear  combination  of  the  cost  indices.  By  minimizing  the  positive 

deviational  variables,  p  and  p  %  from  right  hand  side  values  of 

M3  M4 

zero,  this  goal  can  be  satisfied  since  the  zero-one  goal  program 
attempts  to  select  only  those  projects  with  low  cost  indices. 


Table  5-10 

Application  4:  Model  Constraints 
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Variables 


Row 

No, 

XA 

XB1 

XB2 

XB3 

xc 

XD 

XE1 

XE2 

XE3 

XF 

n . 
i 

Pi 

RHS 

1 

1 

1 

1 

ni 

-Pl= 

1 

2 

1 

1 

1 

n2 

-p2= 

1 

3 

2 

1 

1 

2 

1 

2 

1 

1 

1 

0 

n3 

-P3= 

4 

4 

2 

1 

1 

2 

1 

2 

1 

1 

n4 

"PA= 

5 

5 

1 

1 

2 

1 

2 

1 

n5 

-p5= 

5 

6 

1 

1 

2 

1 

1 

n6 

-p6= 

4 

7 

2 

1 

1 

2 

1 

2 

1 

1 

1 

o 

n7 

-P7  = 

8 

8 

2 

1 

1 

2 

1 

2 

1 

1 

n8 

-Pg  = 

9 

9 

1 

1 

2 

1 

2 

1 

n9 

-P9  = 

9 

10 

1 

1 

2 

1 

1 

n10 

-pio= 

8 

11 

1 

1 

1 

1 

1 

nll 

-pn= 

0 

12 

3 

2 

1 

1 

3 

3 

2 

2 

3 

1 

n12 

-Pl2= 

15 

13 

1 

2 

2 

2 

1 

3 

2 

2 

3 

1 

nl  3 

II 

m 

H 

CL 

\ 

0 

14 

3 

1 

1 
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1 

2 

1 

1 
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1 

n14 
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Priority  6:  Select  those  projects  that  satisfy  an  urgency  of  need 

9  3 

criteria.  This  goal  is  attained  bv  minimizing  p  in  row  15.  ~ 

15 

The  objective  functions  for  the  two  models  are: 

r 

Minimize  a  =  IP  (p +P„)'»  P0(P.+P,+P-+P. ) ;  P^Cp..);  (5. A) 

11-  -  3  4  6  311 


P4^n12^  ’ 

P5(p13+PlA)=  Pfe(p15)r 

r 

Minimize  a  =  |ppPj+P> 

):  P2(P7+P8+P9+P10);  P3(P11); 

(5.5) 

P4(n!2); 

P5(p13+p14);  P6(P15)  * 

The  thesis  model  was  used  to  solve  the  above  problems  and  the 
results  are  reported  in  Table  5-11.  Both  models  include  a  portfolio 
of  projects  that  are  compatible  with  the  development  plan  (goal  3) 
and  satisfy  the  urgency  of  need  criteria  (goal  6).  For  Model  1 
(part  of  the  staff  assigned  to  technical  services',  three  projects 
were  selected  that  have  high  probabilities  of  success  (goal  4).  With 
the  full  staff  available  (Model  II),  two  additional  projects  are 
added  to  the  portfolio  with  the  expected  increase  in  the  measure  of 
success  prohabi ] i t ies .  Neither  model  resulted  in  the  requirement  for 
additional  staff  (achieving  goal  2)  but  at  the  same  time,  it  appears 
that  there  is  an  excessive  amount  of  underutilization  (six  man- 
quarters  of  idle  time  for  Model  I  and  14  man-quarters  for  Model  II). 

Because  of  the  results  concerning  professional  staff  under¬ 
utilization,  it  was  of  interest  to  rerun  both  models  inserting  a  new 
third  priority  goal  of  minimizing  the  underutilization  of  the  research 
staff  in  each  quarter.  This  new  goal  is  satisfied  by  minimizing  the 
negative  deviational  variables  in  rows  3-6  of  Table  5-10  for  Model  I, 
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and  minimizing  the  negative  deviational  variables  in  rows  7-10  for 
Model  II.  The  objective  function  for  the  revised  models,  noted  as 
Models  IA  and  IIA,  which  now  have  seven  priority  goals  (goals  3-6 
for  Models  I  and  II  become  goals  4-7  for  Models  IA  and  IIA),  are 
as  follows: 


Model  IA: 


Model  IIA: 


Minimize  a  = 


Minimize  a  = 


P1(P1+P2);  IVP3+P4+P5+P6)  ’ 

Vn3+VW;  p4(pll); 

P5  ^n12^ ’  P  ^p13+p14^ •  P7  j  * 

P  ( p  +d  )  ;  P  ( p  +p  +p  -bp  )  ; 

1  H1  *  2  ’  2  P7  p8  P9  P10  ’ 

p3(n7+n8+n9+Tii0);  p4(Pn); 

p5(n12);  P6^p13+p14) ’  P7(p15^  ' 


(5.6) 


(5.7) 


The  results  of  the  runs  of  the  thesis  model  are  show  in  Table  5-12. 
With  the  portfolios  selected  for  the  revised  models,  the  staff  under¬ 
utilization  is  significantly  reduced  (two  man-quarters  of  idle  time 
for  Model  IA  and  eight  man-quarters  of  idle  time  for  Model  IIA). 
However,  this  reduction  is  at  the  expense  of  selecting  one  project 
(x  )  which  does  not  support  the  development  plan  goal  for  both  Models 

D  3 

IA  and  IIA,  a  portfolio  of  projects  having  a  lower  total  index  of 
success  probabilities  in  the  case  of  Model  IA,  and  in  the  case  of 
Model  IIA,  a  portfolio  having  a  higher  index  of  costs. 

The  next  step  involved  in  this  application  was  the  performance  of 
a  sensitivity  analysis  on  the  priority  goal  structures.  For  Models  I 
and  II,  which  involved  six  prioritized  goals,  new  problems  were 


i 


i 


i 


Table  5-11 

Application  4:  Portfolios  and 
Goal  Attainment  Levels 
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Model  I 


Model  II 


Projects  Selected:  A,C,E3  A,C,DtE3,F 

Goal  1:  Avoid  Duplication 

of  Projects  B  and  E:  Achieved  Achieved 


Goal  2:  Minimize  Staff 
Overruns : 


Achieved 


Achieved 


(The  Staff  Utilization  is:) 


t=l  4 

t*2  4 

t-3  2 

t-4  2 


8 

6 

4 

2 


Goal  3:  Development  Plan 
Compatibility : 


Achieved 


Achieved 


Goal  4:  Maximize  the 

Indices  of  Success3:  9  13 

Goal  5:  Minimize  Cost  Indices3:  12  19 


Goal  6:  Urgency  of  Heec< 
Criteria : 


Achieved 


Achieved 


a 


Measured  by  the  sum  of  the  indices  of  the  projects  selected. 


formulated  by  switching  goal  A  (maximize  the  indices  of  success)  with 
goal  3  (development  plan  compatibility).  The  solution  to  the  Model  I 
revised  selection  problem  included  the  same  portfolio  as  shown  for 
Model  I  in  Table  5-11.  However,  for  Model  II,  the  solution  suggested 
a  portfolio  including  projects  A,  Bl,  C,  D  and  E3  which  differed  from 
that  shown  in  Table  5-11.  The  associated  goal  attainment  levels  also 
changed,  reflecting  a  reduction  in  the  idle  time  of  the  research  staff, 
a  higher  sum  of  the  probability  of  success  indices,  higher  cost  indexes, 
and  one  project  that  does  not  support  the  development  plan  goal.  Sensi¬ 
tivity  analysis  was  also  performed  on  the  seven  priority  goal  structures 
involved  with  Models  IA  and  IIA.  First,  the  probability  of  success 
goal  was  switched  with  the  development  plan  compatibility  goal  and 
second,  the  goal  involving  minimization  of  costs  was  made  a  higher 
priority  than  the  development  plan  compatibility  and  probability  of 
success  goals.  The  solutions  to  these  formulations  were  identical  to 
those  of  Table  5-12  for  both  models. 

All  of  the  above  results,  which  were  generated  through  the 
application  of  the  thesis  model,  were  presented  to  the  director  of 
research  and  the  practical  utility  of  the  thesis  model  was  discussed. 
Concerning  the  issue  of  practical  utility,  it  was  concluded  that  the 
thesis  model  appeared  workable  in  the  realistic  R&D  environment 
described.  Multiple  objectives,  project  data  and  resource  constraints, 
and  the  interrelationships  of  these  parameters,  could  be  directly 
transformed  into  model  formulations  and  computer  program  input  param¬ 
eters.  Model  output  was  easily  interpreted  and  could  be  related  to  the 
selection  problem  as  constructed  by  the  director  of  research.  Con¬ 
cerning  the  discussion  of  the  model  results,  several  important 
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Table  5-12 

Application  4:  Portfolios  and 
Goal  Attainment  Levels 
(7  Goals) 


Model  IA 


Projects  Selected: 


B3,C,E3 


Goal  1:  Avoid  Duplication  of 

Projects  B  and  E  Achieved 


Goal  2:  Minimize  Staff 
Overruns : 


Achieved 


Goal  3:  Minimize  Staff  Under¬ 
utilization.  The  under  utili¬ 
zation  is: 

t=l 

t-2 

t=3 

t=4 


0 

1 

1 

0 


Goal  4:  Development  Plan 
Compatibility: 


Not  Achieved 
(B3  Selected) 


Goal  5:  Maximize  the 
Indices  of  Success  : 


7 


3 

Goal  6:  Minimize  Cost  Indices  : 


12 


Goal  7:  Urgency  of  Need 

Criteria :  Achieved 


aMeasured  by  the  sum  of  the  indices  of  the  projects 


Model  IIA 

A,B3,C,D,E3 

Achieved 

Achieved 

0 

1 

3 

4 

Not  Achieved 
(B3  Selected) 

13 

21 

Achieved 

selected. 
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conclusions  were  reached.  Specifically,  the  responsibility  of  pro¬ 
viding  technical  services  has  a  direct  impact  on  the  research  mission 
of  the  department.  Here,  the  model  was  useful  in  not  only  identify¬ 
ing  the  obvious  result  that  more  research  can  be  conducted  with  the 
full  staff  available,  but  also  in  illustrating  the  magnitude  of  change 
in  the  attainment  of  the  research  department’s  goals  when  the  resource 
constraint  is  adjusted  towards  the  full  staff  level.  Further,  the 
sensitivity  analysis  provided  by  changing  goal  priorities  and  adding 
goals  was  useful  in  the  identification  of  dominant  projects  (i.e., 
those  projects  which  consistently  appeared  in  each  portfolio  generated). 
Here,  the  model  was  judged  to  be  of  benefit  to  the  decision  maker  by 
reducing  the  subset  of  projects  that  require  further  management 
analysis  for  the  selection  decision.  In  a  more  general  sense,  the 
discussion  of  the  results  provided  a  vehicle  by  which  the  research 
director  and  the  author  could  speak  more  directly  and  with  more 
insight,  about  goal  articulation  and  the  project  selection  decision 
process  within  the  firm.  Both  the  research  director  and  the  author 
concluded  that  this  focused  discussion  was  valuable  in  understanding 
what  was  actually  happening  within  the  firm  and  what  was  desired. 

In  summary,  the  thesis  model  appeared  to  be  a  workable  decision 
aiding  tool  for  a  R&D  project  selection  problem  in  a  corporate  setting. 
It  was  particularly  encouraging  that  the  research  director  expressed 
an  interest  and  made  arrangements  for  additional  work  using  the 
thesis  model. 
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Footnotes 


Validity  refers  to  the  degree  of  similarity  between  model 
results  and  other  reported  good  solutions,  or  the  optimal  solution 
when  known. 

2 

W.  E.  Souder,  "Optimum  Research  and  Development  Models"  (Ph.D. 
Dissertation,  St.  Louis  University,  1970). 

3 

T.  Mandakovic,  "An  Interactive  Model  for  R&D  Project  Selection 
Decision  Making  in  Hierarchical  Organizations"  (Ph.D.  Dissertation, 
University  of  Pittsburg,  1978). 

4 

Aside  from  being  the  most  recent  technique  available,  Manda¬ 
kovic  developed  his  integer  (zero-one)  programming  algorithm  under 
the  tutalage  of  W.  Souder,  perhaps  the  recognized  authority  in  the 
field  of  R&D  project  selection  problem  modeling  and  applications. 
Further,  Mandakovic's  research  includes  algorithm  validity  tests 
using  actual  problem  data.  Thus,  it  provides  a  benchmark  for  com¬ 
parison  with  the  present  research  model. 

^S.  Senju,  and  Y.  Toyoda,  "An  Approach  to  Linear  Programming 
with  0-1  Variables,"  Management  Science,  Vol.  15  (December,  1968), 
pp.  B196-B207 .  '  ~~  ' 

^Y.  Toyoda,  "A  Simplified  Algorithm  for  Obtaining  Approximate 
Solutions  to  Zero-One  Programming  Problems,"  Management  Science, 

Vol.  21  (August,  1975),  pp .  1417-1427. 

^Mandakovic  solved  several  1,000  variable  problems  in  CPU 
times  ranging  from  203  to  310  seconds  on  a  PDP-10  computer  system, 
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l"*The  original  problem  included  8  projects,  a  6  year  planning 
horizon  and  up  to  33  funding  levels  for  each  project  in  each  year 
which  resulted  in  850  project  versions  to  be  examined  in  the 
selection  problem.  Further,  150  problems  were  solved  by  varying 
the  first  year  budget  constraint  from  $  0  to  $  1,500,000  in  increments 
of  $10,000.  For  the  application  reported  in  this  chapter,  the 
major  changes  included  providing  two  funding  levels  for  each  project 
(rather  than  up  to  33)  and  dropping  out  two  projects.  Without  these 
changes,  the  problem  was  too  large  for  the  thesis  model. 
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The  probabilities  associated  with  the  two  funding  levels 
shown  in  Table  5-5  were  those  probabilities  and  funding  levels 
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therefore  chosen  for  comparison  purposes. 

"^In  the  original  problem  reported  by  Rosen  and  Souder,  the 
total  expected  output  goal  was  established  as  15.  This  figure 
was  reduced  to  6.6  by  deducting  the  total  expected  output  values 
for  the  two  projects  dropped  for  this  application. 
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This  goal  was  established  as  55  in  the  original  problem  but 
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in  return  on  expenditure  resulting  from  dropping  out  two  projects. 
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Because  of  the  proprietary  nature  of  the  information  asso¬ 
ciated  with  this  application,  the  textile  firm  elected  not  to 
release  actual  project  descriptions  and,  in  some  cases,  the  actual 
data  parameter  estimates  (i.e.  ,  cost  input  and  probability  of  suc¬ 
cess  parameters).  Accordingly,  a  scaling  process  was  used  for  the 
cost  data  and  probabilities  of  success  which  allowed  the  trans¬ 
mittal  of  the  otherwise  sensitive  information  while  at  the  same  time 
preserved  the  cardinal  scale  relationships  of  the  parameters.  The 
resultant  data  format  did  not  detract  from  the  project  selection  and 
resource  planning  problem  at  hand;  in  fact,  the  general  usefulness 
of  the  model  was  demonstrated  in  that  it  was  shown  to  be  capable  of 
handling  a  wide  variety  of  data  parameters. 


An  alternate  formulation  would  be  to  minimize  the  negative 
deviational  variable  in  a  constraint  equation  containing  a  linear 
combination  of  those  projects  which  are  compatible  with  the  product 
development  plan  and  having  a  large  right  hand  side  value.  Both 
formulations  represent  the  objective  statement  in  that  the  zero-one 
goal  programming  model  attempts  to  select  compatible  projects  and 
not  select  the  incompatible  ones.  However,  the  formulation  shown 
in  Table  5-10  permits  a  more  direct  interpretation  of  the  program 
output . 
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As  with  the  constraint  equation  developed  for  the  priority 
3  goal,  there  are  two  possible  formulations.  The  alternate  con¬ 
straint  equation  would  be  a  linear  combination  of  projects  that 
satisfy  an  urgency  of  need  criteria  with  the  goal  being  to  minimize 
the  negative  deviational  variable  from  a  large  right  hand  side 
value . 


CHAPTER  VI 


CONCLUSIONS  AND  SUGGESTIONS  FOR  FURTHER  RESEARCH 

A  multiperiod,  stochastic,  multiple  objective,  zero-one 
mathematical  programming  model  has  been  presented  for  the  research 
and  development  project  selection  and  resource  allocation  problem. 
Each  feature  of  the  complete  model  has  been  designed  to  incorporate 
realistic  aspects  of  the  R&D  decision  making  process,  thereby 
establishing  a  contribution  to  the  practical  and  theoretical  state- 
of-the-art  in  R&D  project  selection  modeling.  In  the  preceeding 
chapters,  the  model  has  been  demonstrated  to  be  capable  of: 

1)  Selecting  from  a  set  of  potential  research  and  development 
projects  a  portfolio  that  maximizes  the  attainment  of 
multiple,  noncominensurate  objectives  using  a  goal  pro¬ 
gramming  format. 

2)  Incorporating  the  multistage  and  sequential  decision 
making  nature  of  R&D  projects  and  representing  project 
interactions  and  interrelationships  through  decision 
tree  planning  techniques. 

3)  Addressing  the  limitations  of  scarce  multiple  resource 
constraints . 

4)  Representing  mutually  exclusive  and  mutually  dependent 


projects  or  versions. 


5)  Incorporating  discrete  probability  estimates  or  probability 
distributions  for  the  various  parameter  estimates  using 
simulation  techniques. 

6)  Providing  good  solutions  to  medium  scale  problems  in 
relatively  fast  computer  code  processing  times,  thus 
providing  model  users  the  benefit  of  low  cost  multiple 
runs  for  sensitivity  analysis  in  conjunction  with 
planned  changes  in  input  parameters. 

The  model  was  successfully  tested  against  recent  mathematical 
programming  techniques  also  designed  for  the  R&P  project  selection 
problem.  Comparable,  and  in  some  cases,  superior  results  were 
obtained  in  all  of  these  tests,  and  the  added  capa1  Llities  of  the 
thesis  model  were  discussed  and  are  summarized  in  Table  6-1.  In 
addition,  an  appraisal  of  model  feasibility  was  gained  through  an 
actual  application  and  through  presen  tat  ions  to  R&D  laboratory 
directors  and  administrators. 

A  Summary  of  the  Application  of  the  Thesis  Model  in  the  R&D  Project 
Selection  Environment 

The  multistage,  multiple  objective  project  selection  and  resource 
allocation  model,  presented  herein,  is  designed  and  Intended  for  those 
R&D  activities  which  enjoy  a  certain  degree  of  autonomy  with  regards 
to  selecting  the  research  portfolio,  and  that  concentrate  efforts  on 
applied  research  or  development  projects  associated  with  clearly 
defined  end  uses.  On  the  basis  of  the  many  applications  of  the  thesis 
model  carried  out  as  part  of  this  research,  the  following  sequence  of 
activities  for  its  use  by  a  practicing  R&D  manager  is  suggested. 
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The  first  step  in  any  application  of  the  model  is  the  development 
of  the  project  trees  which  provide  the  input  data  for  the  goal  program¬ 
ming  formulation.  Project  or  decision  trees,  which  are  unique  to  the 
particular  organizational  environment,  afford  decision  makers  the 
benefits  of  detailed  project  planning  and  visual  recognition  of  pro¬ 
ject  and  constraint  interrelationships  and  technical  outcome  uncer¬ 
tainty. 

The  next  step  is  the  selection  of  dominant  future  outcome  states 
through  forced  choice  chance  node  selection  or  simulation  techniques. 
Alternatively,  for  small  problems  with  a  manageable  number  of  chance 
nodes,  all  outcome  states  can  be  accounted  for  through  a  stochastic 
representation  as  was  illustrated  in  the  example  problem  presented 
in  Chapter  III. 

The  third  step  in  model  application  involves  the  zero-one  goal 
programming  formulation.  This  step  includes  the  prioritization  of 
the  multiple  objectives  and  development  of  system  and  goal  constraints 
using  input  data  from  the  project  trees  in  order  to  operationalize 
the  objectives.  Following  this  step,  the  zero-one  goal  programming 
algorithm  is  used  to  select  a  portfolio  that  best  satisfies  the  set 
of  multiple  objectives.  Table  6-2  provides  guidelines  for  using  the 
different  versions  of  the  algorithm  depending  upon  the  size  of  the 
problem. 

Finally,  in  those  situations  where  model  users  are  interested  in 
examining  various  strategies  associated  with  changes  in  the  priority 
structure,  resource  constraint  levels,  or  technical  outcome  states, 
multiple  model  runs  were  suggested  for  sensitivity  analysis  purposes. 


Table  6-1 

Model  Capability  Comparisons 


Table  6-2 

Suggestions  for  Use  of  the 
Goal  Programming  Algorithm  Versions 
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Case 


Decision 


1.  If  there  are  less  than 
45  project  variables: 

2.  If  there  are  between  45 
and  75  project  variables: 


3.  If  there  are  between  75 
and  100  project  vari¬ 
ables  : 

4.  If  there  are  more  than 
100  project  variables 
with  clearly  defined 
divisions  or  levels 
providing  up  to  100 
variables  as  their 
input : 

5.  If  there  are  more  than 
100  project  variables 
with  no  disaggregation 
by  division  possible: 


Use  the  variable  selection  and  ex¬ 
change  heuristics  with  full  par¬ 
titioning. 

Use  the  variable  selection  and  ex¬ 
change  heuristics  for  intital  solu¬ 
tion.  In  a  subsequent  run,  partition 
on  those  variables  selected  in  the 
initial  solution. 

Use  the  variable  selection  and  ex¬ 
change  heuristics  without  parti¬ 
tioning. 

Formulate  divisional  subproblems 
and  use  the  appropriate  version  as 
described  in  the  above  cases. 


a.  Use  the  variable  selection 
heuristic  with  or  without  par¬ 
titioning. 

b.  Alternatively,  extract  from  the 
original  problem,  a  problem  of 
tractable  size  and  use  the 
appropriate  version  from  cases 
1  through  3  above.  This  pro¬ 
cedure  has  been  followed  in 
other  applications  involving 
different  models  (see,  for 
example,  Chiu  and  Gear^), 


Multiple  runs  may  also  be  required  for  statistical  analysis  if 
external  parameters  such  as  the  probabilities  of  commercial  success 
or  expected  profits  are  input  as  probability  distributions. 


Suggestions  for  Further  Research 


Direction  of  Future  Research  in  the  R&D  Decision  Making  Environment 

In  a  recent  publication  which  appeared  after  the  present  research 

was  well  underway,  W.  Souder,  the  current  Chairman  of  the  TIMS  College 

of  R&D  Management  and  Director  of  the  Technology  Management  Studies 

Group  at  the  University  of  Pittsburg,  wrote  the  following  assessment 

of  R&D  project  selection  models  and  the  real  world  environment  for 

2 

which  they  are  designed:^ 

Most  models  appear  to  be  constructed  largely  for  the 
single  decision  maker,  rather  than  for  organizational 
decision  making...  the  real  world  complexities  of 
multiple  objectives  and  constraints  are  usually  glossed 
over,  and  the  models  are  formulated  around  single 
objectives  and  single  constraints...  Management  science 
models  reflect  only  the  analytical  aspects  of  project 
evaluation.  But  in  real  world  project  evaluation, 
decisions  are  often  profoundly  influenced  by  a  multi¬ 
tude  of  organizational  and  human  behavioral  factors... 

Three  important  organizational  behavioral  needs 
must  be  satisfied  before  any  project  evaluation  model 
can  be  used  effectively.  First  of  all,  organizat ional 
goals  and  constraints  at  all  levels  of  the  organiza¬ 
tion  must  be  clearly-defined  and  agreed  upon.  They 
are  the  ultimate  standards  for  killing  some  projects 
and  accepting  others. 

Second,  most  project  evaluation  data  are  neces¬ 
sarily  subjective  in  nature.  Unless  a  spirit  of  trust 
and  openness  is  felt  by  the  parties,  it  is  not  likely 
that  such  data  will  be  fully  and  openly  exchanged. 

Third,  for  successful  project  evaluation  a 
minimum  level  of  personal  awareness  is  needed. 

The  present  research  found  a  high  degree  of  consensus  with  these 
observations  and,  indeed,  the  thesis  model  was  specifically  designed 
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with  the  capability  to  handle  the  real  world  complexities  of  multiple 
objectives,  multiple  constraints,  and  uncertainty  in  data  parameters 
and  future  outcome  even..,.  Possessing  these  capabilities,  the  thesis 
model  was  proposed  as  a  decision  aiding  tool  with  practical  utility 
in  the  total  R&D  decision  making  process  which  was  earlier  illus¬ 
trated  and  described  in  Figure  1-1  of  the  introductory  chapter. 
However,  it  is  recognized  that  the  model  is  only  a  part  of  the  process 
and  the  direction  for  future  research  must  be  in  the  upstream  decision 
stages  where  the  model  inputs  are  developed.  That  is,  research  must 
now  concentrate  on  means  of  defining  multiple  objectives,  developing 
accurate  estimates  of  organizational  resources  and  needs,  generating 
future  event  scenarios  and  project  proposal  data  based  on  factual 
information  exchanges,  and  methods  of  achieving  consensus  on  these 
items,  so  that,  in  interaction  with  the  thesis  model,  an  effective 
total  structured  process  is  designed. 

Nominal  group  processes  with  psychometric  Q-Sort  or  pairwise 
comparison  experiments  (discussed  in  Chapter  II)  represent  a  start 
in  this  area,  especially  for  generation  of  the  set  of  multiple 
research  goals.  It  remains  to  test  the  practical  utility  of  a  total 
structured  process  through  an  application  involving  Q-Sort  type 
experiments  for  generation  of  input  data  for  the  thesis  model  and 
feedback  of  output  information  to  the  decision  making  entities. 
Follow-up  analysis  would  be  required  to  determine  if  such  a  total 
process  is  of  enhanced  utility  to  R&D  managers  in  their  determination 
of  a  portfolio  for  research  in  their  organization. 
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Other  Applications  and  Extensions  of  the  Multiple  Objective  Model 

The  thesis  model  was  particularly  designed  for  the  special 
characteristics  of  the  R&D  project  selection  decision  making  process. 
However,  since  the  R&D  project  selection  problem  is  a  special  case 
of  the  more  general  class  of  resource  allocation  problems  and  since 
the  thesis  model  expands  the  capability  of  existing  multiple  objec¬ 
tive  integer  goal  programming  algorithms,  there  exists  a  potential 
for  applications  involving  other  multicriteria  problem  situations 

where  the  decision  variables  represent  "go-no  go"  decisions.  The 

3  4 

zero-base  budgeting  problem  (see,  for  example,  Pvhrr  and  Cheek  )  is 
an  excellent  example  of  this  type  of  problem.  Here,  top  management 
determines  goals  for  organizational  units  and  establishes  expenditure 
guidelines  for  the  budget  year.  Operating  management  develops 
alternative  activity  plans  or  decision  packages  for  achieving  the 
operational  objectives  and  determines  their  expenditure  requirements. 
The  thesis  model  is  suggested  as  an  interactive  decision  aiding  tool 
to  assist  the  organization  in  determining  which  of  the  activity  plans 
should  be  funded  in  order  to  achieve  the  objectives  and  stay  within 
the  established  expenditure  levels.  The  model’s  capability  to  analyze 
various  future  event  scenarios  would  enhance  its  application  in  zero- 
base  budgeting  decision  making.  Similarly,  the  capital  budgeting 
problem  under  capital  rationing  suggests  another  environment  for 
application  of  the  thesis  model. 

Another  area  for  model  application  was  identified  in  the  inter¬ 
view  with  the  director  of  the  Center  for  Applied  Technology  in  the 
nonprofit  research  institute.  There,  the  problem  of  interest  was 
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scientific  staff  resource  allocation  which  is  more  generally  defined 
as  the  capacity  management  problem  in  knowledge  intensive  organiza¬ 
tions.  For  this  particular  organizat ion ,  the  laboratory  directors 
must  periodically  determine  the  allocation  of  staff  members  to 
three  areas:  proposal  development;  marketing  of  proposals  to 
prospective  clients;  and  contract  work  on  proposals  selected  by 
the  clients.  In  this  case,  the  zero-one  goal  programming  model 
can  be  used  by  considering  subunits  of  areas  as  zero-one  decision 
variables  having  different  payoffs  and  resource  utilization  rates. 

Another  direction  for  future  research  concerns  the  zero-one 
goal  programming  algorithm  developed  as  the  solution  technique  for 
the  multiple  objective  model  formulat ions .  The  heuristics  developed 
represent  several  approaches  to  solving  medium  scale  problems. 
Research  in  development  of  other  techniques  that  further  expand  the 
model’s  capability  and  improve  its  efficiency  would  enhance  the 
model’s  potential  for  application  in  multiple  objective  problem 
formulations  having  ordinally  ranked,  non commensurate  goals. 
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APPENDIX  A 


Description  of  the  Computer  Code 
and  Data  Preparation  for  the 
Zero-One  Goal  Programming  Algorithm 


i 


General  Description 

a.  This  program  is  written  in  Fortran  IV  and  includes  variable 
selection,  exchange  and  partition  heuristics  for  the  zero-one 
goal  programming  problem. 

b.  Restrictions 

1)  Only  zero-one  variables  are  permitted. 

2)  System  constraints  are  not  allowed  as  such.  If  any 
exist,  convert  them  to  goal  constraints  by  minimizing 
the  deviational  variables  involved  at  the  first 
priority  level. 

3)  Only  one  deviational  variable  of  a  constraint  equation 
may  be  minimized  in  a  problem.  If  both  deviational 
variables  are  to  be  minimized,  generate  two  constraints 
and  minimize  the  positive  (negative)  deviational  vari¬ 
able  in  the  first  constraint  and  the  negative  (positive) 
deviational  variable  in  the  second. 

4)  Negative  technological  coefficients  in  the  goal  con¬ 
straint  equations  are  not  allowed.  If  any  exist, 
replace  the  respective  variables  with  complementary 
variables  and  add  the  appropriate  number  of  equality 
constraints  to  achieve  the  complementarity.  For 
example,  if  the  coefficient  of  x^  in  one  of  the  goal 
constraints  is  -1,  replace  x^  with  (1  -  x J)  and  add 
the  system  constraint  +  x|  =  1. 

c.  Output  for  this  program  includes  input  data  checks,  the 
variables  selected  in  the  best  solution  and  the  goal  under¬ 
attainment  levels  of  the  best  solution. 
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The  dimensions  for  the  number  of  constraints,  decision 
variables,  and  priority  levels  in  the  computer  code  listed 
in  Appendix  B  are; 

1)  NROVJ,  the  number  of  constraints  (1  1  NROU  <  40). 

2)  NVAR,  the  number  of  variables  (1  5  NVAR  ^  100). 

3)  NPRI,  the  number  of  priority  levels  (1  1  NPRI  f  10). 


Preparation 
Dimension  Card 


Col. 

5  - 

7 

(13) 

Col. 

8  - 

10 

(13) 

Col.  11  - 

13 

(13) 

Objective 

Function 

Cards 

Col. 

1 

(ID 

Col.  5-9  (15) 


Col.  10  -  14  (15) 


Col.  15  -  25  (F11.0) 


The  number  of  constraints. 
The  number  of  variables. 

The  number  of  priorities. 

"1"  if  the  positive  devia¬ 
tion  is  to  be  minimized,  1,2” 
if  the  negative  deviation  is 
to  be  minimized. 

The  row  in  which  the  devia- 
tional  variable  appeared. 

The  priority  level  at  which 
the  deviational  variable  is 
to  be  minimized. 

The  differential  weighting 
factor  of  the  deviational 
variables.  If  the  factor  is 
1.0,  it  must  be  punched. 


Note:  Objective  function  cards  should  be  input  in 

increasing  order  by  row  numbers  (card  for  row  1 
is  first,  row  2  is  second,  etc.) 
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c.  Technological  Coefficients  Cards 

Col-  5  -  9  (15)  The  row  in  which  the  coef¬ 

ficient  appears. 

Col.  10  -  14  (15)  The  column  in  which  the 

coefficient  appears. 

Col.  15  -  25  (F11.0)  The  coefficient's  value. 

Note:  Punch  one  card  for  each  nonEero  coefficient.  The 

program  assigns  a  value  of  0.0  for  any  coefficient  not 
specified.  After  the  last  coefficient  card,  include  a 
trailer  card  with  ,,999n  in  columns  7-9. 

d.  Right  Hand  Side  Value  Cards 

Col.  1-10  (F10.0)  The  right  hand  side  value 

of  the  first  constraint. 

Col.  71  -  81  (F10.0)  The  right  hand  side  value 

of  the  eighth  constraint. 
Note:  If  there  are  more  than  eight  constraint  rows,  con¬ 

tinue  on  the  next  card(s)  with  the  same  format, 
d.  Option  Card 

Col.  1  -  1  (13)  "1M  if  the  variable  selec¬ 

tion  routine  is  to  be  used; 
,,2f'  if  both  the  variable 
selection  and  exchange 


routines  are  to  be  used. 
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e . 


f . 


Partitioned  Subproblems  Dimension  Card 

Col.  1  -  3  (13)  The  number  of  partition  sub¬ 

problems  to  be  solved.  If 
none,  punch  "l"  (this  indi¬ 
cates  use  of  only  the  selec¬ 
tion  and  exchange  heuristics). 

Partitioned  Subproblem  Cards  (one  set  of  cards  for  each 

partitioned  subproblem) 

Col.  1-2  (12)  "l”  if  the  first  variable  is 

considered  for  solution;  "0" 
if  the  first  variable  is 
forced  out  of  solution. 


Col.  79  -  80  (12)  "l"  if  the  40th  variable  is 

considered  for  solution;  "0” 
if  the  40th  variable  is 
forced  out  of  solution. 

Note:  If  there  are  more  than  40  variables,  continue  on 
the  next  card(s),  using  the  same  format.  If  no  partitioned 
subproblems  are  to  be  solved,  include  one  set  of  partitioned 
subproblem  cards  with  each  variable  having  a  value  of  1. 

3.  Additional  Features  and  Suggestions  for  Use  of  the 
Program  Options. 

a.  Write  statements,  enclosed  in  comment  cards,  are  contained 

throughout  the  program.  These  statements  will  provide  interim 
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results  after  each  variable  selection  procedure;  after  each 
iteration  of  the  variable  exchange  routine;  and  at  the  com¬ 
pletion  of  each  partitioned  subproblem  procedure.  To  obtain 
these  interim  results,  repunch  the  appropriate  write  state¬ 
ments  without  the  comment  "C"  entry  in  card  columns  1  and 
76.  Format  statements  are  included. 

b.  The  program  is  currently  written  to  solve  one  problem 
(including  one  or  more  partitioned  subproblems)  at  a  time. 
However,  problem  stacking  is  easily  accomplished  by  changing 
the  counter  control  statement  "IF(IEND.LT.l)  GO  TO  1000” 
(Fortran  statement  number  292)  to  indicate  the  number  of 
stacked  problems.  For  example,  if  three  complete  problems 
are  to  be  solved  in  one  batch  program  submission,  the  counter 
control  statement  should  read  f,IF(IEND  .LT.  3)G0  TO  1000”. 

Note  that  each  stacked  problem  must  include  all  of  the  data 
card^  described  in  paragraph  2  above,  even  if  there  is  some 
replication . 

c.  The  partitioning  procedure  was  designed  specifically  for 
the  general  class  of  zero-one  variable  selection  problems 
having  mutually  exclusive  system  constraints,  i.e.,  select 
either  version  1  or  2  of  project  1,  but  not  both  so  as  to 
avoid  project  duplication.  In  the  above  example,  two  parti¬ 
tioned  subproblems  can  be  solved;  one  with  the  version  1 
decision  variable  forced  out  of  solution  and  the  other  with 
version  2  forced  out.  However,  the  procedure  is  general  and 
any  variable (s)  can  be  selectively  forced  out  of  the  solution 
procedure . 
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d.  The  program  has  been  tested  on  a  variety  of  problems  ranging 
from  3  to  10  priorities,  15  to  62  constraint  rows,  and  6  to 
180  zero-one  decision  variables.  Although  CPU  times  will 
vary  depending  upon  the  structure  of  the  problem  and  the 
particular  computer  system  available,  the  following  sugges¬ 
tions  are  made  for  using  the  program  options  if  running  time 
is  a  consideration. 

1)  If  there  are  less  than  45  zero-one  decision  vari¬ 
ables:  use  the  variable  selection  and  variable 

exchange  option  with  as  many  partitioned  subproblems 
as  dictated  by  the  mutually  exclusive  variable 
system  constraints  and/or  as  desired. 

2)  If  there  are  between  45  and  75  decision  variables: 
use  the  variable  selection  and  exchange  option  for 
an  initial  solution.  If  desired,  partition  on  the 
solution  variables  in  a  subsequent  run. 

3)  If  there  are  between  75  and  100  decision  variables: 
use  the  variable  selection  and  exchange  option. 

4)  If  there  are  more  than  100  variables:  use  the 
variable  selection  option.  The  partitioning 
procedure  may  also  be  employed  with  this  option. 

Note  that  the  array  dimensions  in  the  program  must 
be  changed  if  there  are  more  than  100  variables. 
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APPENDIX  B 


Computer  Code  for  the 
Zero-One  Goal  Programming  Algorithm 
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The  experimental  design  used  in  the  first  experiment  of  the  case 

study  reported  in  Chapter  4  required  testing  the  hypothesis  of  no 

significant  differences  in  mean  benefits  achieved  for  the  portfolio 

strategies  associated  with  the  six  models.  Table  C-l  shows  the 

observed  benefit  values  generated  by  the  simulation  routine,  wilh 

2 

the  means  (x.)  and  the  variances  (s.)  of  each  group,  for  j  =  1,  ...» 

6  groups.  A  one-way  factor  analysis  of  variance  (AN OVA)  test  was 
used  to  determine  if  there  is  a  difference  in  the  benefits  achieved 
for  the  models.  The  ANOVA  table  for  the  observations  is  shown  in 
Table  C-2  and  follows  from  Harnett  and  Murphy.^  By  employing  the  F 
statistic,  the  hypothesis  of  no  difference  in  mean  benefit  values 
is  rejected  Lf  F  observed  is  greater  than  a  critical  F  with  signifi¬ 
cance  level  a,  J-l  and  N-J  degrees  of  freedom.  With  01=.  01,  there 
is  a  significant  difference  between  the  means  of  the  sample  groups. 

To  have  confidence  in  this  conclusion,  the  assumptions  required  by 

ANOVA  must  be  met.  That  is,  for  the  F  statistic  to  perform  correctly, 
2 

there  must  be: 

1)  Independence  of  statistical  error  terms. 

2)  Equality  of  population-error  variances. 

3)  Normally  distributed  populations. 

By  using  the  random  number  generator  through  manipulation  of  the  seed 

values  in  the  manner  described,  independence  within  each  population 

group  is  achieved.  Hartlev's  F  statistic  was  used  to  test  for 

max 

homogeneitv  of  error  variance.  This  test  computed  the  F  with  6 
°  "  max 

(number  of  variances)  and  29  (number  of  observations  within  each  group 

3 

minus  one)  degrees  of  freedom  as  follows: 
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Table  C-l 

Benefits  Attained  for  Models  1-6 


Model  1 

Model  2 

Model  3 

Model  4 

Model  5 

Model  6 

47.24 

57.81 

48.93 

46.64 

41.44 

51.71 

52.47 

53.91 

44.92 

51.40 

45.76 

52.30 

53.86 

49.06 

40.13 

46.70 

46.05 

55.69 

51.61 

52.44 

44.32 

54.22 

43.03 

48.60 

53.24 

47.23 

51.52 

46.47 

46.23 

49.47 

53.75 

51.68 

40.02 

49.86 

45.42 

52.81 

51.17 

51.36 

39.91 

46.04 

42.93 

47.60 

50.52 

58.59 

46.54 

45.04 

50.02 

50.52 

49.64 

48.78 

44.96 

48.60 

51.15 

53.91 

46.34 

51.44 

43.07 

47.93 

41.72 

53.78 

53.44 

53.72 

40.75 

52.11 

46.04 

54.92 

49.07 

48.13 

51.32 

47.84 

46.31 

50.49 

50.63 

51.93 

48.62 

50.62 

44.02 

50.62 

50.99 

47.48 

44.21 

50.45 

45.39 

54.69 

48.  34 

53.94 

42.16 

49.34 

43.39 

52.21 

51.66 

45.46 

48.24 

51.58 

45.86 

54.14 

44.41 

53.81 

47.50 

45.15 

44.35 

51.67 

50.20 

49.17 

48.09 

50.39 

40.85 

52.54 

48.79 

52.03 

45.88 

47.83 

40.72 

53.91 

47.70 

50.86 

47.37 

48.86 

45.19 

53.80 

48.57 

50.27 

42.92 

57.04 

48.26 

52.75 

45.88 

47.46 

44.55 

52.23 

41.04 

53.17 

50.75 

47.05 

42.43 

45.03 

41.92 

53.64 

52.09 

45.24 

43.07 

50.85 

46.88 

47.59 

45.72 

47.85 

47.33 

50.17 

43.05 

49.28 

48.4  9 

53.46 

46.73 

46.97 

39.17 

53.02 

46.76 

45.43 

46.17 

53.03 

41.84 

53.38 

52.35 

52.97 

42.57 

49.37 

47.72 

50.43 

48.90 

45.89 

42.57 

52.12 

43.89 

48.76 

50.31 

48.49 

39.45 

52.69 

45.31 

49.92 

49.830 

50.430 

44.875 

49.552 

44.498 

51.911 

6.537 

12.214 

10.973 

8.543 

7.914 

4.969 

Table  C-2 
ANOVA  Table 


Source  of 
Variation 

(1) 

Sum  of 

Squares  (SS) 

(2) 

Degrees  of 
Freedom  (df) 

(3) 

Mean  Squares 
(l)/(2) 

Between 

Samples  (a) 

1421.716 

J-l  =  5 

284.343 

Error  (b) 
(Within 
samples) 

1483.348 

N-J  =  174 

8.525 

Total 

2905.064 

N-l  =  179 

> 
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,2  /  2 

max  largest/  j  smallest 


=  12.214  /  4.969 


=  2.458 


(c.l) 


The  hypothesis  of  homogeneitv  of  variance  is  rejected  if  F  is 

max 

greater  than  the  tabled  value  for  F  ,  as  developed  from  the  F 

TTicix  max 

4 

distribution.  With  o  =  .01,  the  critical  Fmax  is  3.73  and  therefore, 
the  hypothesis  is  not  rejected.  Finally,  since  the  F  statistic  is 
robust  to  violations  of  nomality,  all  requirements  of  ANOVA  are  met. 

Having  determined  that  there  is  an  overall  difference  in  the 
sample  group  means,  the  next  test  is  to  determine  which  means  are 
different  through  multiple  comparison  analysis.  A  simple  multiple 
comparison  test  developed  by  Tukey  in  1953, called  the  Honestly 
Significantly  Differences  (HSD)  test,  is  used  in  this  study.  For 
the  HSD  test  to  perform  correctly,  the  same  assumptions  required  by 
the  F  statistic  used  in  ANOVA  design  are  also  required.  Since  the 
HSD  test  is  robust  to  violations  of  normality,  this  and  the  other 
two  assumptions  are  again  met.^  The  differences  among  the  means 
of  the  sample  groups  from  Table  Ol  are  arrayed  as  shown  in  Table 
C-3.  A  comparison  between  two  means  is  significant  if  it  exceeds  an 
HSD  which  is  given  by: 

!/2 

HSD  =  q  .  ,  _v  (Mean  Square  Error/  n)  (c.2) 

(ex;  J,  h-J) 

=  4.76  (.533)  =  2.537 


The  value  of  q  is  obtained  from  the  distribution  of  the  studentized 
range  statistic. ^  For  this  test,  a=  .01  and  the  degrees  of  freedom 


I  X 


r 


X6 

x2 

1 

4 
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Table  C-3 

Differences  Among  Mean 
Benefit  Attainment  Levels 


*2 

*4 

*3 

*5 

1.481 

2.081 

2.359 

7.036* 

7.413* 

- 

.600 

.878 

5.555* 

5.932* 

- 

.278 

4.955* 

5.332* 

_ 

4.677* 

5.054* 

377 


are  J=  6,  and  N-J=  174.  The  values  marked  with  an  asterisk  in  Table 
03  indicate  a  significant  difference  between  corresponding  means. 


Footnotes 


D.  L.  Harnett,  and  L,  L.  Murphy,  Introductory  Statistical 
Analysis  (Reading  Mass.:  Add ison-Wesley  Publishing  Co.,  Inc., 

1975),  Chap.  9. 

2 

R.  E.  Kirk,  Experimental  Design:  Procedures  for  the  Behavioral 
Sciences  (Belmont,  Cal.:  Brooks/Cole  Publishing  Co.,  1968),  p.  61. 

3 

H.  0.  Hartley,  "The  Maximum  F-ratio  as  a  Short-cut  Test  for 
Heterogeneity  of  Variance,"  Biometrika,  Vol.  37  (1950),  pp.  308-312. 

Virk,  Table  D.10,  p.  536. 

5Ibid.,  p.  88. 

6Ibid. 

7  lb  id . ,  Table  D.7,  pp.  531-532. 
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The  experimental  design  used  in  the  second  experiment  of  the  case 
study  reported  in  Chapter  4  required  testing  the  hypothesis  of  no 
significant  differences  in  the  benefits  achieved  for  the  portfolio 
strategies  associated  with  the  two  models  versus  the  alternate 
hypothesis  that  the  benefit  values  of  model  1A  are  greater  than  those 
of  IB.  Table  D-l  shows  the  observed  benefit  values  generated  by  the 
simulation  routine.  The  statistical  technique  used  to  test  this 
hypothesis  is  the  t  test  for  differences  between  matched  samples."^ 

This  test  requires  computation  of  the  difference  scores  of  the 
benefit  values  for  each  model  as  shown  in  Table  D-l.  The  hypothesis 
of  no  difference  between  the  samples  is  rejected  if  the  observed  t 
statistic,  of  the  form: 

tn_i  =  t29  =  (D  -  0)  /  (sD  /  (n)  ' *)  =  5.902  (d.l) 

is  greater  than  a  critical  ratio  of  a  one  sided  test  with  an  a  level 
2 

of  .01.  In  this  case,  the  critical  t  is  2.765  and  the  hypothesis  of 
no  difference  is  rejected. 


Table  D-l 

Difference  Scores  on 
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the  Benefits  Attained 


do6  $) 


(i) 

Model  1A 

(2) 

Model  IB 

Di 

Observation 

Benefits 

Benefits 

(1)  -  (2) 

1 

54.24 

53.88 

.36 

2 

49.84 

48.69 

1.15 

3 

51.37 

47.30 

4.07 

4 

48.31 

46.01 

2.30 

5 

50.18 

47.36 

2.82 

6 

42.58 

48.80 

3.78 

7 

54.74 

52.71 

2.03 

8 

46.33 

44.96 

1.37 

9 

50.22 

46.87 

3.35 

10 

50.41 

50.39 

.02 

11 

49.35 

49.68 

-  .33 

12 

50.92 

49.02 

1.90 

13 

53.62 

49.66 

3.96 

14 

54.49 

50.69 

3.80 

15 

49.56 

46.99 

2.57 

16 

53.24 

52.12 

1.12 

17 

51.85 

49.95 

1.90 

18 

49.93 

48.02 

1.91 

19 

48.79 

50.07 

-1.28 

20 

52.79 

48.43 

4.36 

21 

50.76 

49.99 

.77 

22 

54.12 

55.01 

-  .89 

23 

50.92 

47.62 

3.30 

24 

52.39 

52.83 

-  .49 

25 

47.59 

43.46 

4.13 

26 

54.14 

5  3.96 

.18 

27 

50.19 

46.00 

4.19 

28 

45.03 

44.68 

.35 

29 

46.28 

45.96 

.32 

30 

47.30 

45.70 

1.60 

Mean : 

50.716 

48.895 

1.821 

Standard  Deviation 


2.633 


2.952 


1.690 


Footnotes 


^D.  L.  Harnett,  and  J.  L.  Murphy,  Introductory  Statistical 
Analysis  (Reading,  Mass.:  Addison-Wesley  Publishing  Co.,  Inc., 
1975),  pp.  363-366. 

o 

A  one  sided  test  is  used  since  the  alternate  hypothesis  is 
directional. 
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